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EXECUTIVE  SUMMARY 

The  Assessment  of  Military  Multitasking  Performance: 

Validation  of  a  Dual-task  and  Multitask  Protocol 
Award  number  W81XWH-12-2-0070;  lONovember  2015 


Background 

Concussed  personnel  commonly  experience  persistent  cognitive,  postural,  and  vestibular  deficits 
as  well  as  disabling  headaches,  hearing  or  visual  impairments,  dizziness,  and  sleep  dysfunction.  These 
symptoms  can  impair  duty  performance  and  disrupt  a  SM’s  ability  to  safely  reintegrate  with  the  unit. 
Drawing  from  the  sports  concussion  literature,  military  clinicians  often  base  return  to  duty  (RTD) 
determinations  on  results  from  clinical  measures  of  cognition,  balance  testing,  and  symptom  self-report 
measures  during  exertional  testing.  These  existing  RTD  measures  are  problematic  because  they 
collectively  lack  valid  norms,  have  poor  face  validity,  or  have  not  yet  been  validated  among 
Warfighters.  Research  aims  were  designed  to  begin  validation  of  a  functional,  performance-based 
assessment  protocol  to  address  gaps  identified  by  the  Gray  Team  II  Report  of  the  Chairman  of  the  Joint 
Chiefs  (April  201 1)  and  the  DTM  09-033,  by  developing  a  functional  assessment  to  guide  RTD 
decisions  following  mTBI  that  can  be  administered  both  in  the  deployed  and  garrison  based 
environments.  Initial  1-year  (Sept  2009-Oct  2010)  MRMC  funding  allowed  collection  of  stakeholder 
and  expert  specifications  and  literature  review  to  specify  an  assessment  concept  and  develop  a 
preliminary  set  of  9  tasks  based  on  dual-task  and  multitask  assessment  methods.  . 

Using  the  previously  developed  set  of  nine  performance-based  assessment  tasks  called  the 
Assessment  of  Military  Multitasking  Performance  (AMMP),  funds  were  awarded  by  the  Army  Medical 
Research  Materiel  Command  for  preliminary  validation  of  this  assessment  battery  using  service 
members  with  residual  symptoms  following  concussion/mild  traumatic  brain  injury  (mTBI)  and  healthy 
control  (HC)  subjects.  The  four  specified  aims  of  this  project  included:  1)  Further  specify  and  refine  a 
set  of  dual-tasks  and  multitasks  with  procedures  for  test  administration,  2)  Evaluate  interrater  reliability 
for  each  of  the  dual-task  and  multitask  metrics  using  healthy  control  and  service  members  with 
diagnosed  mTBI  undergoing  rehabilitation  for  residual  symptoms  (known  groups),  3)  Evaluate  and 
interpret  hypothesized  correlations  between  scores  on  neurobehavioral  and  sensorimotor  domain  tests 
and  scores  on  AMMP  dual-tasks  and  multitasks  in  healthy  control  SMs  and  SM  with  mTBI  to  determine 
convergent  validity,  and  4)  Evaluate  the  ability  of  dual-task  and  multitask  test  items  to  discriminate 
between  healthy  control  SM  and  SM  with  mTBI  residual  symptoms  undergoing  rehabilitation. 

This  report  describes  the  methods  and  outcomes  of  this  validation  protocol. 

Methods 

Using  healthy  control  subjects  from  the  US  Army  Research  Institute  of  Environmental  Medicine 
(Natick,  MA)  and  both  healthy  control  and  subjects  with  mTBI  residual  symptoms  from  Fort  Bragg,  NC, 
three  phases  of  data  collection  were  conducted  involving  an  iterative  task  refinement  process  for  the 
AMMP  including  feasibility  considerations  and  the  use  of  repeated  inter-rater  reliability  testing  to 
improve  task  metrics,  administration  and  scoring  instructions.  The  six  tasks  that  remained  in  the  AMMP 
battery  after  this  refinement  process  were  then  used  to  evaluate  convergent  and  discriminate  validity. 
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Key  Accomplishments  and  Findings 

1.  Achieved  clinically  acceptable  inter-rater  reliability;  all  test  metrics  for  each  of  the  six  tasks 
remaining  in  the  AMMP  battery  have  interrater  reliability  above  0.90  for  dual-tasks  and  above  0.88 
for  multitasks  except  for  several  metrics  on  errors  and  cues  which  were  then  not  used  to  evaluate 
between  group  differences. 

2.  Based  on  a  minimum  correlation  of  0.3  for  evaluation  of  hypothesized  relationships  between 
standard  neurocognitive  measures  and  AMMP  task  metrics;  these  hypotheses  were  supported. 
Although  small  (almost  all  below  0.4),  expected  correlations  were  demonstrated  for  most  tasks  with 
the  exception  of  the  Patrol-exertion  task. 

a.  Small  correlations  may  be  due  to  complexity  of  the  tasks  with  multiple  factors  affecting 
performance.  In  addition,  the  mTBI  group  was  not  tested  at  the  time  of  evaluating  AMMP 
tasks  for  several  assessments  including  Neurobehavioral  Assessment  Battery  and 
Comprehensive  Trail  Making  Test  tests,  which  may  have  been  a  confounder  given  lag  times 
between  neurocognitive  testing  (obtained  from  Womack  TBI  Clinic  chart)  and  AMMP 
battery  testing. 

b.  Dynamic  Visual  Acuity  Testing  (http://www.micromedical.com/Products/)  was  administered 
as  a  behavioral  measure  of  gaze  stability  and  a  proxy  for  vestibular  function  testing.  The 
DVAT  identified  only  3  subjects  with  mTBI  who  demonstrated  clinically  significant 
impairments  in  gaze  stability  (>  2  lines  lost)  during  rapid  head  impulses  (>  120  degrees/sec) 
relative  to  static  visual  acuity  during  a  head  stationary  condition.  This  clinical  reference 
point  was  consequently  insufficient  for  use  as  a  correlational  measure  for  AMMP  metrics 
intended  to  target  vestibular  vulnerabilities. 

3.  Subjects  with  mTBI  appear  to  have  more  difficulty  than  HC  with  cognitive  tasks  especially  when 
overlaid  on  physical  challenge: 

a.  Dual-task  interference  for  the  cognitive  task  when  tested  during  the  motor  task  was  able  to  distinguish 
HC  from  mTBI  groups  in  the  Load  Magazine-Radio  Chatter  and  Instrumented  Stand  and  Walk-Grid 
Coordinates  Dual  Task. 

b.  Metrics  focusing  on  executive  function  (cognitive)  vulnerabilities  demonstrated  the  ability  to 
distinguish  groups  in  the  Charge  of  Quarters  Duty  task.  A  combination  of  exertional, 
cognitive  and  reaction  time  requirements  in  the  Patrol-exertion  task  resulted  in  symptom 
report  of  reduced  visual  clarity  and  slower  reaction  time  metrics  which  distinguished  mTBI 
compared  to  HC. 

4.  Other  than  reduced  peak  velocities  during  180  degree  turns  during  normal  walking  as  measured  by 
inertial  sensors,  and  reaction  times  during  the  mulitask,  Patrol-exertion,  AMMP  physical 
performance  metrics  alone  were  insufficient  to  distinguish  groups,  leading  to  speculation  that  for  the 
most  part  subtle  differences  may  require  inertial  sensors  or  other  laboratory  equipment  to  identify 
physical  performance  dissimilarities  in  elite  Warfighters  that  are  healthy  versus  concussed. 

5.  Using  1)  the  AMMP  task  metrics  that  discriminate  groups,  2)  PCL-C  scores  (PTSD  screen),  and  3) 
self-assessment  for  readiness  to  return  to  combat  duty,  predictive  modeling  through  logistic 
regression,  discriminant  analysis  and  decision  tree  analysis  (CHAID)  yielded  successful 
classifications  of  HC  and  mTBI  between  80.8%  and  91.7%  of  cases.  Accepting  a  number  of 
limitations  such  as  an  inability  to  use  cases  with  missing  data,  these  methods  show  promise  in 
identifying  specific  factors  and  AMMP  metrics  that  provide  the  most  information  and  that  may  be 
further  investigated  for  RTD  decision-making. 

6.  Exploratory  analyses  using  time  series  analysis  methodologies  (principal  component  analysis  (PCA)) 
is  demonstrating  the  potential  to  correctly  categorize  healthy  control  and  subjects  with  mTBI.  These 
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exploratory  findings  require  further  investigation  with  the  current  data  set  to  develop  a  theoretical 
framework  for  their  interpretation  and  association  with  known  mTBI  related  vulnerabilities.  . 

7.  There  is  likely  an  optimal  balance  of  symptom  report,  impairment  and  functional  metrics  that  can  be 
modeled  to  optimally  discriminate  between  ready  and  non-duty  ready  personnel. 

8.  An  AMMP  Training  Manual  and  Training  Modules  (PowerPoint  and  Video)  are  included  in  appendices  and 
under  separate  files.  For  each  of  the  six  AMMP  tasks,  these  materials  provide  1)  Task  Description  and  Set 
Up,  2)  Examiner  Instructions  and  Script,  3)  Score  Sheet,  4)  Scoring  Guide,  5)  Task  specific  testing  materials. 
The  AMMP  Training  Manual  and  Modules  are  designed  to  enable  prospective  AMMP  examiners  to 
understand,  visualize,  and  practice  the  administration  of  the  6  AMMP  tasks. 


Future  Research  Recommendations  and  Next  Lines  of  Inquiry 

Next  steps  in  research  include  both  general  “gap  area”  themes  as  well  as  specific  research  question  to 
promote  development  and  validation  of  individual  tasks.  Both  are  presented  in  this  section  however  next 
steps  in  research  will  admittedly  be  informed  by  DoD/  stakeholder  needs,  availability  of  test  subjects 
within  FORSCOM  or  in  medical  readiness  platforms  in  the  MEDCOM.  Research  prioritization  may  also 
be  a  function  of  specific  interests  and  expertise  among  clinical  researchers  in  study  design  and 
administration  related  to  the  AMMP  tasks. 

AMMP  BATTERY 

We  recommend  creation  of  an  AMMP  Steering  Committee  during  the  bridge  period  following  this 
current  initial  validation  effort  and  next  steps,  to  assist  with  monitoring  and  potentially  coordinating 
further  validation  studies.  Some  focus  areas  include: 

•  Gap  1:  Tailoring  AMMP  Tasks  to  address  different  types  of  RTD  decision  making  (Acute 
vs.  Chronic  Injury);  Time  since  injury:  Short  (days)  vs.  Long  (months).  The  current  AMMP 
battery  includes  performance  tests  that  could  be  used  to  screen  SM  in  austere,  far  forward 
settings  (to  inform  early  RTD  decision  making  after  acute  injury)  and  other  AMMP  tasks  likely 
better  suited  to  measure  performance  in  established  practice  environments  (Hospital  Settings). 
Follow  on  research  should  be  pursued  to  identify  optimal  administration  points  in  the  continuum 
of  care  and  to  validate  their  use  in  appropriate  practice  environments.  Considerations  may 
include  practical  constraints  to  administration  (space  or  maximal  time  requirements  to  set  up  and 
administer)  or  equipment  factors  (durability  of  instrumented  systems  (e.g.,  computers  or 
accelerometers)  to  reliably  function  in  austere  environments). 

•  Gap  2:  Standardized  Scoring:  In  current  form,  each  individual  AMMP  test-task  is  assessed 
based  on  participant  performance  on  a  series  of  task  specific  component  metrics  (e.g.,  time  to 
perform,  number  of  errors,  number  of  items  correctly  identified,  etc.).  Thus,  overall  task 
performance  is  based  on  calculating  performance  on  a  series  of  sub-component  metrics  which 
have  not  been  normalized  to  a  standardized  scoring  system  within  the  battery.  Investigators  see 
significant  value  in  the  development  of  a  scoring  system  based  on  percentages  or  on  a  Likert 
based  system  that  specifies:  a)  better  than  average,  b)  average,  or  c)  worse  than  average,  will 
contribute  to  the  clinical  utility  of  these  assessment  tasks.  This  approach  would  allow  relative 
distinctions  between  performances  on  individual  AMMP  tasks  and  provide  more  specific 
feedback  on  a  participant’s  relative  strengths  and  challenges. 
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•  Gap  3:  Phase  II  Discriminant  Validity:  To  meet  with  the  primary  RTD  gap  area  of 

characterizing  readiness  to  RTD,  the  research  team  would  propose  testing  in  a  cohort  consisting 
entirely  of  SM  with  history  of  concussion  to  discriminate  between  those  who  are  duty  ready 
(despite  recent  history  of  concussion)  from  non-duty  ready  personnel  who  may  appear  functional 
but  are  for  some  reason,  non-duty  ready  (e.g.,:  still  symptomatic,  subtly  impaired  in  one  or  more 
physiologic  systems;  or  unable  to  perform  Warrior  Tasks  or  Battle  Drills  to  standard). 

•  Responsiveness  to  Change:  If  individual  AMMP  tasks  are  to  be  used  to  measure  progress  toward 
RTD  readiness  as  a  result  of  rehabilitation  (intervention),  research  to  evaluate  metric  responsiveness 
to  change  is  essential.  This  will  dictate  the  need  for  alternate  forms  of  several  of  the  AMMP  tasks. 

•  Predictive  modeling  using  AMMP  task  metrics  should  focus  on  predicting  ‘Successful  Return  to 
Duty’.  Success  for  different  RTD  roles  must  be  defined  and  studies  that  follow  the  Service 
Member’s  RTD  experience  must  occur  to  generate  the  ultimate  objective  of  the  AMMP,  to  develop  a 
predictive  model  for  successful  RTD.  Factors  which  may  be  evaluated  might  include  retention  and 
promotion,  annual  evaluations,  and  use  of  health  care  resources  for  continuing  mTBI  symptoms. 

AMMP  SPECIFIC  TASK  DEVELOPMENT 

•  Specific  AMMP  tasks  may  have  greater  sensitivity  in  certain  patient  groups  (e.g.  ISAW  Grid  in 
acutely  concussed  personnel,  rolling  component  of  Run  Roll  Aim  for  those  reporting  post  traumatic 
dizziness  as  a  clinical  complaint).  Studies  to  identify  AMMP  task  use  based  on  patient  complaint 
profile  are  needed. 

•  Using  an  up  tempo  video  (this  may  be  a  ready-made  video  that  provides  a  busier  and  more 
challenging  cognitive  load),  evaluate  the  Patrol-Exertion  task  at  point  of  RTD  in  established 
concussion  care  programs  following  rehabilitation  (given  exertion  component  and  high  face  validity) 
(Candidate  sites:  WOMACK,  NC;  Fort  Campbell  KY).  Additionally,  develop,  test,  and  validate 
alternate  version  Patrol-exertion  videos  to  include  more  complicated  scenarios  requiring  greater 
cognitive  focus  to  allow  use  of  the  Patrol-exertion  task  in  testing  for  responsiveness  to  change  and 
response  to  treatment  interventions. 

•  Two  of  the  AMMP  dual-tasks  (ISAW-Grid  and  Load  Magazine-Radio  Chatter)  may  be  considered 
for  use  in  their  current  state  on  a  limited  basis  in  military  rehabilitation  settings  to  provide  further 
information  from  military  clinicians  on  their  utility  and  relationship  to  RTD  success/failure.  Data 
from  this  clinical  use  should  be  evaluated  to  provide  further  validation  of  these  dual-task  metrics  to 
provide  useful  clinical  information  on  patient  performance  and  progress. 

•  Develop,  test,  and  validate  alternate  version(s)  of  the  CQ  Duty  task  to  allow  testing  for 
responsiveness  to  change  and  response  to  treatment  interventions  focused  on  mTBI  vulnerabilities 
including  executive  dysfunction  and  impaired  attention  and  memory. 

•  Using  the  ISAW-Grid  and  a  modification  of  the  Illinois  Agility-Word  List  task:  Obtain  serial 
measures  of  dual  task  cognitive  load-dynamic  stability  performance  in  a  cohort  of  SMs  (Active  Duty 
Cadets  or  acutely  concussed  Special  Forces  SM)  pre-  and  post-concussion  from  boxing  or  other 
sports  injury  to  establish  recovery  curves  of  static  postural  stability,  dynamic  stability,  agility, 
attention  and  working  memory  in  highly  functional  and  highly  motivated  personnel  (USMA) 

o  ISAW-Grid  task  at  baseline,  24  hours,  72  hours,  96  hours,  1  week,  9  and  14  days  to 
follow  recovery  from  acute  concussion  and  once  normal  recovery  curves  are  determined, 
to  evaluate  responsiveness  to  interventions  such  as  the  graded  return  to  activity  protocol. 
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o  Illinois  Agility-cognitive  task  (like  Grid-coordinates)  at  baseline,  assuming  asymptomatic 
(headache  and  dizziness  resolved)  and  again  at  5,  7,  9  and  14  days) 
o  Validate  similar  dual-task  performance  study  in  civilian  athlete  cohorts  (e.g.,  academia, 
NCAA,  NFL). 

•  Use  Load  Magazine-  Radio  Chatter  test  to  measure  performance  in  AD  Soldiers  under  stressed 
temperature  conditions  (extreme  hot/  extreme  cold)-  Cross  validate  other  military  functional  tasks 
requiring  bi-manual  dexterity  (USARIEM) 

•  Based  on  findings  from  the  current  inertial  sensor  data  of  the  Illinois  Agility-Packing  List  and  Run 
Roll  Aim  tasks,  consider  further  studies  with  inertial  sensor  instrumentation  to  evaluate  the  high 
level  mobility  and  subtle  mobility  deficits  of  Warrior  athletes  potentially  with  a  combination  of 
motor  tasks  (floor  to  stand  transitions  and  combat  rolls)  that  challenge  vestibular  vulnerabilities. 

•  Encourage  additional  studies  to  translate  the  use  of  Dual-task  and  Multitask  assessments  with  face 
validity  to  civilian  tests. 


Conclusions 

The  successful  completion  and  key  findings  of  this  study  provides  ample  reason  to  support  on-going 
research  to  refine  and  apply  the  AMMP  battery  for  evaluating  the  SM  with  mTBI  with  regard  to  readiness 
for  successful  return  to  duty.  Defining  parameters  that  identify  what  successful  RTD  looks  like  is  a  key  next 
step  for  successful  prediction.  We  believe  that  AMMP  tasks  may  be  used  as  well  in  the  civilian  setting  with 
minimal  modification. 
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ABBREVIATIONS 


AMMP:  Assessment  of  Military  Multitasking  Performance 
BI:  Brain  Injury  (Mild  Traumatic  Brain  Injury) 

COG  =  Cognitive  Priority 

CQ  Duty:  Charge  of  Quarters  Duty 

CTMT:  Comprehensive  Trail  making  Test 

DNI:  Do  Not  Instruct 

DoD:  Department  of  Defense 

DVAT:  Dynamic  Visual  Acuity  Test 

DVBIC:  Defense  and  Veterans  Brain  Injury  Center 

FDD:  Front  Desk  Duty 

FFT:  Fast  Fourier  Transform 

FORSCOM:  US  Army  Forces  Command 

HC:  Healthy  Control 

HR:  Heart  Rate 

HRV:  Human  Research  Volunteer 
ICC:  Intraclass  Correlation  Coefficient 
IED:  Improvised  Explosive  Device 
IQR:  Interquartile  Range 
IRR:  Interrater  Reliability 

ISAW-Grid:  Instrumented  Stand  and  Walk  -Grid  Coordinates 

MEDCOM:  US  Army  Medical  Command 

MET:  Multiple  Errands  Test 

MOB:  Mobility  Priority 

MOS:  Military  Occupational  Specialty 

mTBI:  Mild  Traumatic  Brain  Injury 

NAB:  Neurobehavioral  Assessment  Battery 

NAB_DB:  Neurobehavioral  Assessment  Battery:  Digits  Backward 

NAB-D:  Neurobehavioral  Assessment  Battery:  Number-Letters  Part  D 

NSI:  Neurobehavioral  Symptom  Inventory 

PCL-C:  PTSD  Checklist-Civilian  Version 

PLS-DA:  Partial  Least  Squares  -  Discriminative  Analysis 

RTD:  Return  to  Duty 

Rx:  Reaction 

SM:  Service  man/men  or  woman/women 
SRT:  Simple  Reaction  Time 
SVM:  Support  Vector  Machine 
TOH:  Tower  of  Hanoi 

USARIEM:  U.S.  Army  Research  Institute  of  Environmental  Medicine 
USAARL:  US  Army  Aeromedical  Research  Laboratory 
WAMC:  Womack  Army  Medical  Center 
WRAT:  Wide  Range  Achievement  Test  Reading 
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INTRODUCTION 


The  Assessment  of  Military  Multitasking  Performance  (AMMP)  is  a  battery  of  functional  dual-tasks  and 
multitasks  based  on  military  activities  that  target  known  sensorimotor,  cognitive,  and  exertional 
vulnerabilities  after  concussion/mild  traumatic  brain  injury  (mTBI).  The  AMMP  was  developed  to  help 
address  known  limitations  in  post  concussive  return  to  duty  (RTD)  assessment  and  decision  making. 

The  purpose  of  this  research  was  to  validate  the  AMMP  battery.  The  scope  of  research  included:  1) 
finalizing  a  set  of  tasks  that  met  feasibility  standards,  2)  insuring  inter-rater  reliability  for  scoring  of 
final  task  metrics,  3)  evaluate  convergent  and  discriminate  validity  using  correlation  of  AMMP  task 
metrics  to  standard  neurocognitive  tests  and  4)  test  known  group  validity  by  comparing  performance  of 
tasks  for  healthy  controls  (HC)  and  soldiers  with  mTBI.  AMMP  is  intended  for  use  in  combination  with 
other  metrics  to  inform  duty-readiness  decisions  in  Active  Duty  Service  Members  following  concussion. 


BODY 

RESEARCH  ACCOMPISHMENTS  BY  STUDY  AIMS 


Introduction 

This  section  first  describes  the  design  and  study  populations  from  both  the  U.S.  Army  Research  Institute 
of  Environmental  Medicine  (USARIEM)  and  from  Fort  Bragg  for  the  validation  studies  on  the  Assessment  of 
Military  Multitasking  Performance  (AMMP).  Extensive  demographic  information  is  provided,  particularly  for 
the  subject  pool  from  Fort  Bragg.  A  brief  summary  of  subject  characteristics,  a  discussion  of  findings  on  test 
order  bias  and  on  our  measure  of  fatigue  is  included  at  the  end  of  this  initial  section,  as  is  a  brief  description  of  the 
AMMP  battery  of  tasks.  Following  this,  the  findings  are  presented  for  each  of  the  four  project  aims.  The  final 
section  of  this  report  includes  findings  that  result  from  exploratory  analysis  of  the  data  on  AMMP  task  metrics 
that  can  distinguish  the  two  study  populations. 

Study  design 

This  study  involved  a  convenience  sample  case-control  measurement  study  involving  test  construction 
and  evaluation  using  a  target  population  of  concussion/mild  traumatic  brain  injury  (mTBI)  subjects  and  healthy 
controls  (HCs).  This  measurement  development  study  involved  three  sequential  phases  which  used  interrater 
reliability  (IRR)  findings,  informal  and  qualitative  feedback  from  raters  and  subjects,  as  well  as  logistical  and 
practical  properties  evaluation  by  the  test  developers  to  drive  task  revision.  The  first  phase  of  this  study  involved 
HC  subjects  at  USARIEM  and  focused  on  iterative  IRR  testing  to  drive  task  refinement.  The  final  phase  2  phases 
of  this  study,  which  were  conducted  with  both  HC  and  subjects  with  mTBI  residual  symptoms  at  Fort  Bragg, 
included  several  project  aims.  The  studies  at  Fort  Bragg  included  a  study  to  address  IRR  for  a  subset  of 
individuals  with  mTBI,  followed  by  IRR  for  both  HC  and  mTBI  subjects;  correlational  analyses  to  relate  AMMP 
components  to  impairment  based  neurobehavioral  tests  (convergent  validity);  and  known  groups  comparison  of 
AMMP  ability  to  differentiate  participants  with  mTBI  from  HC  participants  (known  groups  to  support  construct 
validity). 

Participants 

The  following  inclusion/exclusion  criteria  were  used  to  evaluate  potential  subjects  for  all  phases  of  this  study. 

Inclusion  Criteria 
Healthy  Controls 
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•  Service  men  or  women  (SM),  active  duty  that  remain  with  their  unit  and  are  eligible  for  deployment,  age 
18  to  42  years  of  age. 

Subjects  with  mTBI 

•  Service  men  or  women  (active  duty)  receiving  care  for  mTBI-related  symptoms  within  the  Department  of 
Defense  (DoD)  system.  For  the  current  protocol,  this  reflects  subjects  being  seen  via  Womack  Army 
Medical  Center  (WAMC)  TBI  Pipeline  as  a  proxy  for  ongoing  or  persistent  mTBI  symptoms  that  require 
intervention. 

•  Ages  18  to  42  years  of  age. 

•  mTBI  documented  in  medical  record  by  subject’s  physician  or  TBI  trained  clinician  at  time  of  study 
enrollment  based  on  a  positive  mechanism  of  injury  and  altered  or  lost  consciousness  at  time  of  injury. 

•  Clinical  presentation  >  2  weeks  and  not  >  2  years  after  documented  date  of  injury  (mTBI  producing). 

•  Vision  (corrected  or  uncorrected)  sufficient  for  unassisted  reading  and  performance  of  everyday  personal 
tasks  and  independent  community  ambulation. 

•  SM  presents  with  hearing  (without  amplification)  adequate  for  engaging  in  close-range  personal  or 
telephone  conversation. 

•  SM  must  be  able  to  perform  functional  activities  at  moderate  (12  to  14  on  Borg  Rating  of  Perceived 
Exertion)  levels  for  up  to  10  minute  testing  sessions  with  intermittent  breaks  as  needed  and  tolerate  a  total 
test  time  of  120  to  180  minutes  with  frequent  breaks. 

Exclusion  Criteria  (both  groups) 

•  SM  diagnosed  with  severe  brain  injury  or  who  has  a  sustained  penetrating  head  injury. 

•  SM  has  documented  activity  restrictions  incompatible  with  safe  performance  of  AMMP  test  demands. 

•  SM  presents  with  any  medical  or  behavioral  health  condition  that  render  him/her  unable  to  perform 
moderate  exertion  for  up  to  10  minutes,  quickly  changing  positions  including  getting  up  from  the 
floor,  sitting  for  up  to  30  minutes,  lifting  up  to  20  pounds.  During  screening,  the  physical 
requirements  of  the  study  will  be  summarized  and  SM  will  be  asked  if  they  wish  to  continue. 
(Personnel  with  physical  profile  scores  reflecting  PUL  scores  (Army  Profile  System)  of  other  than 
“1”  will  be  reviewed  for  eligibility.  If  the  SM  agrees  to  participate  and  has  difficulty,  any  task  not 
tolerated  will  be  discontinued. 

•  SM  has  hearing  deficits  that  render  him/her  unable  to  communicate  effectively  in  a  clinical  or 
community  setting. 

•  SM  has  functional  deficits  in  visual  integrity,  acuity,  efficiency  or  processing  that  render  him  or  her 
unable  to  perform  one  or  more  of  the  tasks  described  to  him/her  in  the  AMMP  testing  battery. 

Relative  Exclusion  Criteria 

•  Following  informed  consent,  as  part  of  screening,  the  Study  Coordinator  will  describe  the  AMMP 
tasks  and  then  ask  the  SM  if  he/she  would  like  to  continue.  Given  the  activities  of  the  AMMP  battery, 
we  anticipate  an  individual  with  significant  pain  complaints  might  self-select  to  discontinue  study 
participation.  Likewise,  the  Patrol-Exertion  task  includes  viewing  a  virtual  scene  that  was  created  by 
the  developers  of  Virtual  Afghanistan,  so  individuals  with  significant  PTSD  could  decide  to 
discontinue  study  participation. 

Once  enrolled,  the  PTSD  Checklist-Civilian  Version  (PCL-C)  and  the  Visual  Analog  Pain  Scale  was  completed 
by  all  participants  to  allow  for  consideration  of  baseline  stress  and/or  pain  as  a  co-variate,  but  scores  on  these 
measures  were  not  used  to  exclude  participation  in  data  collection. 

Recruitment  and  Subject  Characteristics 

The  subjects  from  USARIEM  were  recruited  by  briefings  and  recruitment  materials  and  included  both 
permanent  party  and  human  research  volunteers  (HRVs)  who  were  stationed  at  USARIEM.  Demographics  for 
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the  20  HC  volunteers  are  found  in  Table  A.  The  2nd  round  of  testing  on  12  HC  subjects  conducted  at  USARIEM 
(for  2  revised  tasks)  used  a  subset  of  the  original  20  volunteers  and  specific  demographic  information  was  not 
repeated  from  this  subject  pool. 

The  subjects  from  Fort  Bragg  were  recruited  in  several  ways.  HC  subjects  were  recruited  by  convenience 
sampling  of  volunteers  from  Fort  Bragg  Special  Operations  and  the  528  Sustainment  Brigade,  subjects  who 
responded  to  recruitment  postings  placed  at  fitness  centers  and  cafeterias  around  the  base,  as  well  as  volunteers 
from  the  Defense  and  Veterans  Brain  Injury  Center  (DVBIC)  in-processing  briefings.  These  DVBIC  briefings 
occurred  on  an  almost  daily  basis  as  part  of  the  standard  informational  training  provided  to  all  soldiers  in¬ 
processing  to  Fort  Bragg  after  transfer  from  other  duty  stations.  Subjects  with  mTBI  were  recruited  from  the 
clinical  population  receiving  rehabilitation  services  at  the  WAMC  TBI  Clinic  using  inclusion/exclusion  criteria  as 
described  in  above.  A  total  of  54  HC  and  54  subjects  with  mTBI  were  recruited  from  Fort  Bragg,  not  all  subjects 
answered  all  intake  questions  and  not  all  subjects  were  able  to  complete  all  AMMP  tasks.  Subjects  filled  in  an 
extensive  intake  form  including  the  PCF-C  and  a  Neurobehavioral  Symptom  Inventory  (NSI)  symptom  checklist. 
Our  study  team  made  an  error  in  transferring  the  PCF-C  checklist  to  the  intake  form  and  only  19  of  the  22 
questions  were  asked  of  most  subjects.  Summary  tables  to  describe  our  study  population  at  Fort  Bragg  are  seen 
below  (Tables  B1-B6). 


Table  A.  USARIEM  Subjects:  Demographics  \  Service  \  Symptoms 


VARIABLE 

HEALTHY  CONTROLS  (N=20)* 

Age:  median(range) 

25  (19-32) 

Sex 

Female 

9  (45%) 

Male 

11  (55%) 

Ethnic  Background 

Black  /  African  American 

8  (40%) 

Caucasian 

7  (35%) 

Hispanic  /  Latino 

3  (15%) 

Other 

2  (10%) 

First  Language  English 

15  (75%) 

Education 

High  School 

7  (35%) 

Some  college  or  Associate  Degree 

6  (54.5%) 

Bachelor’s  Degree 

4  (20%) 

Graduate  or  Professional  Degree 

3  (15%) 

WRAT  Reading  level 

Percentile:  median(range) 

53  (0-99) 

Grade 

5-8 

3  (15%) 

9-12 

4  (20%) 

>12 

13  (65%) 

Months  of  Service:  median  (range) 

19  (4-106) 

Total  Symptom  Score:  median  (range) 

2  (0-16) 

*  Version  2  of  2  AMMP  tasks  were  evaluated  on  a  HC  Subset,  N=12 

WRAT  =  Wide  Range  Achievement  Test  Reading 

Table  Bl.  Fort  Bragg  Subjects:  Demographics  \  Service  \  Symptoms 


Data  Item  (#HC,#mTBI) 

HC 

mTBI 

p-value,  method 

Age  (years)  (mean(SD), 
median(range)  (54,54) 

30.4  (6.0) 

30  (19-42) 

27.7  (5.9) 

25.5  (19-42) 

0.007,  Mann- Whitney  U 
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Data  Item  (#HC,#mTBI) 

HC 

mTBI 

p-value,  method 

Sex  (54,54) 

3  (5.6%) 

0.073,  Fisher  Exact 

Female 

10  (18.4%) 

51  (94.4%) 

Male 

44  (81.5%) 

Ethnicity(54,54) 

Asian/Pacific 

3  (5.6%) 

5  (9.3%) 

0.023,  Chi-Square  Exact 

Biracial 

0  (0%) 

1  (1.9%) 

Black/ African  American 

16  (29.6%) 

4  (7.4%) 

Hispanic/Latino 

6(11.1%) 

9  (16.7%) 

Native  American 

0  (0%) 

2  (3.7%) 

White 

28  (51.9%) 

33  (61.1%) 

Other 

1  (1.9%) 

0  (0%) 

Education(54,54) 

Bachelor’s  Degree 

19  (35.2%) 

9  (16.7%) 

0.008,  Chi-Square  Exact 

Graduate/Professional  Degree 

7  (13.0%) 

1  (1.9%) 

High  School  graduate/GED 

7  (13.0%) 

10(18.5%) 

Some  college  or  Associate  Degree 

21  (38.9%) 

34  (63.0%) 

English  1st  Language(54,54) 

45  (83.3%) 

44  (81.5%) 

1 ,  Chi-Square  Exact 

Handedness  (54,54) 

Both 

3  (5.6%) 

2  (3.7%) 

1 , Chi-Square  Exact 

Left 

5  (19.3%) 

6(11.1%) 

Right 

46  (85.2%) 

46  (85.2%) 

Pain  now  (YES)  (54,54) 

6(11.1%) 

32  (59.3%) 

<0.001,  Fisher  Exact 

For  those  with  pain  now:  Pain  level 
now  (Mean(SD),  median(range)), 

3.3  (1.8), 

3.0  (1.6), 

0.549,  Mann- Whitney  U- 

(6,32) 

3.5  (1-6) 

2(1-8) 

Exact 

Pain  last  week  (YES)  (54,54) 

16  (29.6%) 

43  (79.6%) 

<0.001,  Monte-Carlo  Exact 

For  those  with  pain  last  week:  Pain 
level  last  week  (Mean(SD), 

3(1.8) 

3  d-8) 

<0.001,  Mann-  Whitney  U 

median(range),  (16,43) 

2(1-7) 

2  (1-9) 

Energy  level  (mean  (SD), 

6.7  (1.2) 

6.5  (1.1) 

0.512,  Mann  Whitney  U 

range)(53,54) 

7  (5-10) 

7  (3-10) 

ADHD  (54,54) 

1  (1.9%) 

11  (20.4%) 

0.004,  Fisher  Exact 

Learning  Disability  (54,54) 

1  (1.9%) 

4  (7.4%) 

0.363,  Fisher  Exact 

Table  B2.  Fort  Bragg  Subjects:  Military  History 


Data  Item 

HC 

mTBI 

p-value,  method 

Service  Time  (yrs)  (54,54) 

8.9  (5.6), 

6.1  (5.2), 

0.004,  Mann-Whitney  U 

Mean  (SD),  median  (range) 

8.2  (0.3-23.3) 

3.6  (0.8-23.0) 

Branch  (54,54) 

Army 

52  (96.3%) 

53  (98.1%) 

1 ,  Exact  Chi-square 

Air  Force 

1  (1.9%) 

1  (1.9%) 

Navy 

1  (1.9%) 

0  (0%) 

Active  Duty  (54,54) 

52  (96.3%) 

54  (100%) 

0.495,  Fisher  Exact 

Pay  Grade  (54,54) 

E1-E6 

26  (48.1%) 

44  (81.5%) 

0.002,  Exact  chi-square 

E7-E9 

9  (16.7%) 

1  (1.9%) 

WO-l-WO-5 

2  (3.7%) 

2  (3.7) 

01-03 

15  (27.8%) 

6(11.1%) 

04-010 

2  (3.7%) 

1  (1.9%) 

Primary  MOS  Category  (54,54) 

Force  Sustainment 

27  (50.0%) 

21  (38.9%) 

0.012,  Exact  Chi-square 

Operation 

9  (16.7%) 

2  (3.7%) 
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Operations  Support 

18  (33.3%) 

31  (57.4%) 

Deployed  to  Iraq/Afghanistan  (54,54) 

39  (72.2%) 

38  (70.4%) 

0.832 

Total  Deployments  Iraq  (26,15) 

1 

9  (34.6%) 

8  (53.3%) 

0.265,  Exact  chi-square 

2 

13  (50.0%) 

4  (26.7%) 

3 

2  (7.7%) 

3  (20.0%) 

4 

2  (4.9%) 

0  (0%) 

Total  Deployments  Afghanistan  (27,31) 

1 

17  (63.0%) 

25  (80.6%) 

0.170,  Exact  chi-square 

2 

8  (29.6%) 

3  (9.7%) 

3 

2  (7.4%) 

1  (3.2%) 

5 

0  (0.0%) 

1  (3.2%) 

7 

0  (0.0%) 

1  (3.2%) 

MOS  =  Military  Occupational  Specialty 

Table  B3.  Fort  Bragg  Subjects:  Injury  history 


Data  Item 

HC 

mTBI 

p-value,  method 

Sustained  an  injury  that  prevented  duty  >  24 

hours  during  past  12  months  (54,54) 

4  (7.4%) 

39  (72.2%) 

<0.001,  Fisher  Exact 

If  injury  “yes”-  number  of  injuries  (4,39) 

1-2 

3  (75.0%) 

27  (69.2%) 

0.601,  Exact  chi-square 

3-5 

0  (0%) 

8  (20.5%) 

6  or  more 

1  (25.0%) 

4  (10.3%) 

Symptoms  past  week  (54,54) 

Fisher  Exact 

Memory  problems/lapses 

4  (7.4%) 

37  (68.5%) 

<0.001 

Balance 

1  (1.9%) 

19  (35.2%) 

<0.001 

Bright  light 

2  (3.7%) 

26  (48.1%) 

<0.001 

Irritability 

4  (7.4%) 

33  (61.1%) 

<0.001 

Headache 

5  (9.3%) 

43  (79.6%) 

<0.001 

Sleep  Problems 

6(11.1%) 

39  (72.2%) 

<0.001 

Sustained  a  concussion  ever?  (1  or  more) 

(54,54) 

14  of  54  (26.0%) 

53  of  54  (98.1%)* 

<0.001,  Fisher  Exact 

Number  of  head  injuries/concussions  (14,53) 

1-2 

7  of  13  (53.8%) 

27  (50.0%) 

3-5 

5  of  13  (38.5%) 

16  (29.6%) 

0.603,  Exact  chi-square 

>5 

1  of  13  (7.7%) 

11  (20.4%) 

*One  subject  in  the  mTBI  group  did  not  answer  this  question  on  the  intake  form. 

Table  B4.  Fort  Bragg  Subjects:  Other  reported  health  issues 


Data  Item 

HC 

mTBI 

p-value,  method 

Seen  health  care  provider 

Combat  stress  (53,50) 

PTSD  (53,50) 

Anxiety  (53,50) 

Depression  (53,50) 

Percent  of  54 

3  (5.6%) 

3  (5.6%) 

7  (13.0%) 

7  (13.0%) 

Percent  of  54 

5  (9.3%) 

9  (16.7%) 

13  (24.1%) 

13  (24.1%) 

Based  on  responders 
0.480,  Fisher  Exact 

0.067,  Fisher  Exact 

0.139,  Fisher  Exact 

0.136,  Fisher  Exact 

Average  sleep  /  day 

(mean  (SD),  median  (range))  (53,54) 

6.2  (1.4),  6  (4-10) 

5.4  (1.4),  5  (3-8) 

0.008,  Mann  Whitney  U 
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Case  138  excluded  from  HC  due  to  value  of 

18 

Hours  sleep  past  24  hours  (mean  (SD), 
median  (range))  (54,54) 

6.5  (1.4),  6.5  (4-9) 

5.9  (2.1), 6  (0-15) 

0.023,  Mann  Whitney  U 

Number  of  caffeinated  drinks  (54,54) 

0 

21  (38.9%) 

26  (48.1%) 

0.124,  Exact  chi-square 

1 

18  (33.3%) 

14  (25.9%) 

2 

7  (13.0%) 

12  (22.2%) 

3 

5  (9.3%) 

1  (1.9%) 

>3 

3  (5.6%) 

1  (1.9%) 

Sleep  aid  (54,54) 

3  (5.6%) 

16  (29.6%) 

0.002,  Fisher  Exact 

Hearing  loss  since  enlisted  (54,54) 

17(31.5%) 

30  (55.6%) 

0.019,  Fisher  Exact 

Increased  sensitivity  to  sound  (54,54) 

7  (13.0%) 

26  (48.1%) 

<0.001,  Fisher  Exact 

Ringing  in  ears  (54,54) 

15  (27.8%) 

37  (68.5%) 

<0.001,  Fisher  Exact 

Difficulty  hearing  in  a  noisy  room  (54,54) 

14  (25.9%) 

37  (68.5%) 

<0.001,  Fisher  Exact 

Table  B5.  Fort  Bragg  Subjects:  Standardized  tests-patient  reported  on  intake  form 


Data  Item 

HC 

mTBI 

p-value,  method 

NSI  total  (mean  (SD),  median  (range)) 
Incomplete  NSI  (45,42) 

Complete  NSI  (9,12) 

5.3  (9.2),  2  (0-53) 

2.4  (2.6),  3  (0-6) 

21.1  (12.1),  17.5  (2-52) 
29.6  (17.0),  29  (4-57) 

Mann-Whitney 

<0.001 

<0.001 

NSI  Domain  Vestibular  (54,54) 

0.5  (1.2),  0(0-7) 

3.1  (2.4),  3  (0-9) 

<0.001,  Mann- 
Whitney 

NSI  Domain  Somatosensory  (54,54) 

1.7  (3.5),  0.5  (0-19) 

6.6  (4.3),  5  (0-17) 

<0.001,  Mann- 
Whitney 

PCL-C  (mean  (SD),  median  (range) 

(54,54) 

22.1  (8.0),  19(17-63) 

31.8  (14.4),  30  (0-73) 

<0.001,  Mann- 
Whitney 

PCL-C  Checklist>45  (54,54) 

1  (1.9%) 

7  (13.0%) 

0.060,  Exact  chi- 
square 

Ready  to  return  to  duty  (54,54) 

49  (90.7%) 

27  (50.0%) 

<0.001, Exact  chi- 
square 

Note:  NSI  =  Neurobehavioral  Symptom  Inventory-22 

NSI  scores  will  be  presented  separately  for  those  with  and  without  missing  items  16-18. 

The  two  domain  scores  include  all  subjects  since  they  do  not  include  the  missing  items. 
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Table  B6.  Fort  Bragg  Subjects:  Neurocognitive  and  Dynamic  Visual  Acuity  Results 


Test  (#HC,#mTBI) 

HC 

mTBI 

p-value,  method 

Tower  of  Hanoi  (#HC,  #mTBI) 
(Mean  (SD),  median  (range) 

#  moves  (53,34) 

Error  percentile  (53,34) 

Total  Errors  (53,34) 

10.0  (2.1),  9.8  (7.2-15.4) 
0.04  (0.03),  0.04  (0-.13) 

1.8  (1.6),  2  (0-7) 

10.5  (2.5),  10  (7.2-19.6) 
0.05  (.04),  0.04(0-0.18) 
2.6  (3.2),  2  (0-18) 

0.223,  Mann- Whitney 
0.479,  Mann- Whitney 
0.315,  Mann- Whitney 

Test  of  Memory  (54,54) 
Insufficient  effort  (%) 

1  (1.9%) 

8  (14.8%) 

0.002,  Exact  chi-square 

NAB  Numbers  and  letters 
(Mean  (SD),  median  (range) 

Part  A,  percentile  (54,44) 

Part  B,  percentile  (54,44) 

Part  C,  percentile  (54,44) 

Part  D,  percentile  (54,44) 

28.8  (28.2),  21  (9-96) 

46.2  (28.2),  46  (9-96) 

31.3  (21.2),  24(1-84) 

34.8  (27.8),  29  (9-96) 

23.8  (28.4),  13  (9-99.9) 

27.8  (23.7),  19.5  (9-96) 
29.4  (25.5),  19.5  (1-99) 

27.4  (27.6),  21  (9-92) 

0.131,  Mann- Whitney 
0.001,  Mann- Whitney 
0.210,  Mann- Whitney 
0.060,  Mann- Whitney 

NAB  Digits 

(Mean  (SD),  median  (range) 
Forward,  percentile  (54,44) 
Backward,  percentile  (54,44) 

36.0  (29.3),  24  (9-95) 

50.7  (28.6),  46  (9-92) 

29.4  (29.5),  19.5  (9-95) 
36.0  (27.2),  31  (9-90) 

0.247,  Mann- Whitney 
0.010,  Mann- Whitney 

Trail  Making  (percentile) 

(54,43) 

Composite 

Percentile 

46.2  (9.6),  46.5  (29-68) 
39.7  (29.5),  37.5  (1-96) 

40.2  (9.0),  41  (17-62) 
22.9  (22.1),  18  (9-89) 

0.005,  Mann- Whitney 
0.006,  Mann- Whitney 

WRAT  Reading  (mean  (SD), 
median  (range) 

(>12.9  converted  to  13) 
Standardized  (53,54) 

Grade  level(53,54) 

102.1  (12.5),  101  (76-134) 
11.9(1.7),  12.7  (5.1-13) 

95.7  (10.7), 95.5  (70-119) 
10.8  (2.4), 11.6  (3.8-13) 

0.007,  Mann- Whitney 
0.004,  Mann- Whitney 

DVAT 

Right  line  lost  (52,46) 

0 

1 

2 

3 

4 

Left  line  lost(52,46) 

0 

1 

2 

3 

5 

Missing=2 

31  (59.6%) 

10  (19.2%) 

11  (21.2%) 

0  (0%) 

0  (0%) 

32  (61.5%) 

11  (21.2%) 

9  (17.3%) 

0  (0%) 

0  (0%) 

Missing=8 

14  (30.4%) 
18(39.1%) 

11  (23.9%) 

2  (4.3%) 

1  (2.2%) 

14  (30.4%) 

17  (37.0%) 
12(26.1%) 

2  (4.3%) 

1  (2.2%) 

0.010,  Exact  chi-square 

0.010,  Exact  chi-square 

NAB  =  Neurobehavioral  Assessment  Battery 

WRAT  =  Wide  Range  Achievement  Test  Reading 

DVAT  =  Dynamic  Visual  Acuity  Test  using  Micromedical  Instrumentation,  >  2  lines  lost  considered  abnormal  with  this 
instrumentation 

General  Findings 

Test  Populations 

A  review  of  the  extensive  demographic  tables  above  shows  that  the  HC  and  mTBI  groups  are  dissimilar 
in  a  number  of  characteristics.  The  HC  subjects  are  older,  more  educated,  have  higher  ranks  and  more  years  of 
military  service.  Qualitatively,  to  the  AMMP  research  team,  this  difference  appeared  to  be  the  result  of 
recruitment  and  testing  issues;  HC  subjects  with  higher  ranks  were  more  likely  to  have  control  of  their  schedules, 
thus  allowing  them  to  agree  to  study  participation  and  to  be  able  to  present  themselves  for  testing.  Interested 
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potential  HC  subjects  of  lower  rank  often  were  called  away  by  Command  requirements  and  could  not  be  tested 
when  scheduled.  The  pool  of  subjects  with  mTBI  commonly  had  multiple  required  medical  appointments  in  a 
given  week  and  thus  their  Command  officers  were  more  likely  to  be  accustomed  to  having  potential  subjects  be 
away  from  their  duty  station  for  appointments.  This  appeared  to  result  in  a  greater  likelihood  that  Command  for 
mTBI  subject’s  would  allow  participation  in  AMMP  testing,  thus  resulting  in  testing  inclusive  of  a  younger  pool 
of  subjects.  The  subject  group  with  mTBI  report  more  symptoms,  more  issues  with  PTSD  and  more  hearing 
issues  all  of  which  are  common  in  a  pool  of  active  duty  military  personnel  with  a  history  of 
concussion/mTBI.  Both  HC  and  mTBI  groups  had  equivalent  numbers  of  deployment  experiences.  Interpretation 
of  findings  should  take  in  to  account  these  differences  in  subject  demographics. 

Test  Order  Bias 

During  testing  at  Fort  Bragg,  an  attempt  was  made  to  rotate  the  order  of  task  administration  to  avoid 
having  test  results  regularly  affected  by  fatigue  or  by  testing  proximity  to  another  task  that  might  evoke 
distressing  symptoms.  A  consistent  order  of  AMMP  task  administration  was  not  found.  A  simple  evaluation  of 
the  distribution  of  administration  time  was  done  to  look  at  whether  a  task  was  administered  in  the  first  half  (1-3) 
or  second  half  (4-6)  of  tasks.  No  evidence  of  bias  was  found  for  any  of  the  6  tasks  being  administered  in  any 
specific  order. 

Measure  of  Fatigue  using  Simple  Reaction  Time  (SRT) 

Using  SRT  as  a  measure  of  fatigue,  our  findings  indicate  that  the  HC  and  mTBI  were  similar  in  SRT  at 
baseline,  and  subjects  with  mTBI  are  significantly  slower  than  HC  at  the  end  of  AMMP  testing  (Table  C).  This 
may  be  indicative  of  a  greater  level  of  fatigue  compared  to  HC  after  completion  of  the  AMMP  task  set.  The 
contribution  of  fatigue  to  task  scores  cannot  be  determined  although  attempts  were  made  to  rotate  the  order  of 
testing  (see  Test  order  bias  above)  to  avoid  having  fatigue  affect  any  one  AMMP  task  score  more  than  another. 


Table  C.  Comparison  of  Pre-  and  Post-testing  SI 

IT  as  Fatigue  Measure. 

Metric 

(#HC,  #mTBI) 

HC 

Mean(SD) 

Median(Range) 

mTBI 

Mean(SD) 

Median(Range) 

p-value,  method 

Baseline  (msec) 

(51,33) 

279.4(38.6) 

275.3(223.7-453.5) 

301.8(77.5) 

282.2(208.3-555.4) 

0.664,  Mann- Whitney 

End  of 

AMMP(msec)(5 1 ,34) 

258.0(41.3) 

254.4(214.2-509.5) 

324.2(116.8) 

314.2(213.2-620.8) 

0.006,  Mann-Whitney 

Change  in  reaction 
time(end-baseline) 

-21.4(31.4) 

-17. 7(-113. 5-55.9) 

22.4(85.5) 

-4.0(-95.8-293.7) 

0.017,  Mann  Whitney 

Brief  Descriptions  of  AMMP  Tasks 

The  six  tasks  that  compose  the  AMMP  were  developed  using  either  a  dual-task  or  multitasking  paradigm.  All 
tasks  employ  observational  metrics  and  some  tasks  also  utilize  inertial  sensor  data  to  characterize  SM 
performance. 

AMMP  Multitasks 
Charge  of  Quarter  Duty 

The  SM  is  challenged  to  develop  and  execute  a  work  plan  for  completing  an  array  of  interleaving  tasks  (supply 
inventory,  PVC  foot  stool  assembly,  providing  information  to  superiors,  prospective  memory  tasks)  associated 
with  his/her  hypothetical  assignment  to  Charge  of  Quarters  Duty. 

Run-Roll-Aim 
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The  SM  completes  a  high  level  mobility  task  with  multiple  maneuvers  while  carrying  a  simulated  weapon. 
Maneuvers  are  cued  by  a  computer  screen  with  a  handheld  remote  controlled  slide  advancer.  The  task  requires  a 
rapid  start,  avoiding  a  “trip  wire”  obstacle,  performing  a  3-5  second  rush,  combat  rolling,  searching  for  visual 
targets  through  simulated  weapon  scope,  rapid  lateral  dodging  and  back  pedaling. 

Patrol-Exertion 

The  SM  is  challenged  to  gather  intelligence  in  a  recorded  video  depicting  a  virtual  Afghanistan  patrol 
environment  while  reporting  observed  IED  markers  based  on  a  briefing  provided  at  the  beginning  of  the  video. 
The  SM  then  uses  the  information  to  answer  specific  questions  from  memory  at  the  end  of  the  patrol  video.  The 
SM  will  perform  continuous  step-ups  on  an  exercise  step  at  an  intensity  of  65-85%  of  HR  maximum  throughout 
the  activity  while  being  monitored  for  effort  level  via  a  Polar  HR  monitor  and  performance  observation.  The  SM 
will  be  wearing  a  combat  helmet,  eye  protection,  and  be  carrying  a  simulated  M16  weapon  equipped  with  a 
trigger  switch  connected  via  Bluetooth  to  a  computer  configured  to  record  reaction  time  (RT).  The  SM  is 
required  to  press  the  switch  each  time  a  beep  tone  stimulus  is  heard  throughout  the  video  as  a  measure  of  RT 
during  a  divided  attention  multitask. 

AMMP  Dual-tasks 

Illinois  Agility  Test  -  Packing  List 

The  Illinois  Agility  Test  requires  running  distances  of  30’  with  rapid  direction  changes  and  navigation  of 
obstacles  in  a  serpentine  pattern  during  the  middle  part  of  the  obstacle  course.  A  memory  task  is  also  completed. 
Then  both  the  agility  task  and  the  memory  task  are  performed  at  the  same  time.  Accuracy  of  memory  recall  and 
time  to  complete  the  agility  task  are  measured  in  single  and  dual-task  conditions. 

Instrumented  Stand  and  Walk  -  Grid  Coordinates 

The  SM  is  challenged  to  perform  the  Instrumented  Stand  and  Walk  (ISAW)  test  (developed  by  APDM)  which 
includes  instrumented  and  timed  assessment  of  quiet  standing  for  30  seconds,  assessment  of  dynamic  stability 
during  walking  for  two  7  m  (23  foot)  lengths  with  a  180  degree  turn  at  midpoint.  The  SM  will  next  memorize  an  8 
digit  alphanumeric  grid  coordinate  provided  within  the  context  of  a  simulated  patrol  mission  brief  and  report  the 
exact  sequence  back  to  the  examiner  after  45  seconds.  Finally,  both  the  ISAW  and  the  grid  memorization  tasks 
will  be  performed  simultaneously.  Accuracy  of  grid  coordinate  recall,  postural  sway  area,  gait  path  variability, 
and  time  to  complete  the  ISAW  (i.e.  gait  speed)  will  be  measured  in  single  and  dual-task  conditions. 

Load  Magazine  -  Radio  Chatter 

SM  completes  a  relatively  automatic  manual  task  choosing  from  a  bin  of  mixed  size  dummy  rounds  (5.56  and 
7.62  caliber)  and  loading  5.56  caliber  training  rounds  into  magazines  as  fast  as  possible  both  in  a  single  and  a 
dual-task  condition.  The  dual-task  condition  requires  monitoring  radio  communication  and  verbally  announcing 
when  radio  chatter  is  relevant  to  scenario  instructions. 


AIM  1:  FURTHER  SPECIFY  AND  REFINE  A  SET  OF  DUAL  AND  MULTITASKS  WITH 
PROCEDURES  FOR  TEST  ADMINISTRATION 

Description  of  Iterative  Process 

This  measurement  development  study  involved  three  sequential  phases  which  used  IRR  findings, 
informal  and  qualitative  feedback  from  raters  and  subjects,  as  well  as  logistical  evaluation  by  the  test  developers 
of  practical  task  factors  to  drive  task  revision.  Specifically,  the  team  integrated  lessons  learned  during  testing  in 
an  iterative  manner  over  successive  phases  of  testing  to  improve  face  validity  of  the  tasks  and  the  quality  of 
subject  performance  data.  Investigators  also  worked  to  decrease  test  burden  on  participants  by  decreasing  test 
administration  time.  Other  considerations  during  task  development  and  refinement  included  consideration  of  the 
cost  and  durability  of  equipment,  and  testing  space  requirements.  Revisions  included  dropping  or  modifying  tasks 
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with  poor  IRR  or  poor  face  validity,  refinement  of  task  administration  and  scoring  instructions,  and  improving 
operational  definitions  and  scoring  metrics. 

Phase  I  involved  HC  volunteers  tested  at  USARIEM  in  Natick,  MA.  During  Phase  la,  we  evaluated  the 
feasibility  and  IRR  of  the  original  9  tasks  comprising  the  original  version  of  the  AMMP1,2.  Three  of  the  original 3 
5  multitasks  were  eliminated  and  salient  components  were  refined  and  consolidated  into  one  multitask  called  the 
Charge  of  Quarters  (CQ)  Duty  Task  (see  Figure  1).  The  3  original  multitasks  demonstrated  poor  IRR  due  to 
unclear  operational  definitions  of  success  and  failure  on  task  components  (Figure  1),  and  difficulty  observing  all 
components  of  each  task.  One  of  the  4  original  dual-tasks,  the  Step  initiation-Stroop  task  was  dropped  after  this 
phase  of  testing  in  favor  of  dual-task  assessments  with  greater  face  validity1,2  and  due  to  concerns  about  durability 
of  instrumentation.  For  Phase  lb,  feasibility  and  IRR  of  2  tasks  were  evaluated  on  12  HC  subjects,  the  new 
multitask,  CQ  Duty  and  a  revised  SALUTE  multitask  was  modified  to  incorporate  improvised  explosive  device 
(IED)  marker  reporting.  In  a  later  phase  of  this  study,  the  SALUTE  multitask  was  modified  to  incorporate  the 
reaction  (Rx)  time  dimension  of  the  eliminated  Step  initiation-Stroop  task. 

Phase  II  involved  subjects  with  mTBI  tested  at  Fort  Bragg,  NC.  The  goal  of  Phase  II  was  to  evaluate  the 
IRR  and  feasibility  of  the  revised  AMMP  tasks  named  in  the  middle  section  of  the  Figure  in  subjects  with  mTBI 
residual  symptoms.  This  phase  of  testing  took  place  at  Fort  Bragg,  NC  with  subjects  primarily  in  military 
occupation  specialties  related  to  combat  and  combat  support.  This  phase  was  another  iteration  of  reliability  and 
feasibility  evaluation  in  subjects  on  an  active  Army  base. 

The  goal  of  Phase  III  was  to  evaluate  the  IRR  and  feasibility  of  the  AMMP  tasks  after  the  final  revisions 
to  all  tasks  were  completed.  A  total  of  26  both  HCs  and  subjects  with  mTBI  were  evaluated  in  this  phase  which 
was  completed  over  a  several  month  time  period. 

The  final  result  of  this  iterative  development  process  is  a  set  of  6  tasks,  3  dual-tasks  and  3  multitasks 
which  were  evaluated  for  their  ability  to  discriminate  known  groups  (HCs  and  subjects  with  mTBI  symptom 
complex)  and  for  correlation  to  standard  neurocognitive  tests  as  a  means  to  evaluate  convergent  and  discriminate 
validity. 
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Phases  la  USARIEM  (20  HC),  lb  USARIEM  (12  HC) 


•  A-Bag  Packing 

•  Duty  Roster 

•  ISAW-Grid 

•  Illinois  Agility-Packin 
List 

•  Load-Magazine 
Radio  Chatter 

•  Pack-Ship 

•  Run-Roll-Aim 

•  Salute-Exertion 

•  Step  Initiation-Stroop 


TASKS  DROPPED 


Multitasks 

•  Difficult  to  observe 

•  Poor IRR 

•  Large  task  burden 


Step  Initiation-Stroop 

•  Low  face  validity 

•  ^Equipment  durability 


UPDATED  TASK(S) 


CQ  Duty 
(Multitask) 


SALUTE-Exertion 
IED  marker 
reporting  added 


Phases  lla  FORT  BRAGG  (12  mTBI),  Mb  FORT  BRAGG  (7  mTBI) 


Phases  III  FORT  BRAGG  (19  mTBI,  7  HC) 

ISAW-Grid 

Illinois  Agility-Packing  List 
Load  Magazine-Radio  Chatter 
Run-Roll-Aim 
Patrol-Exertion 
CQ  Duty 


Figure  1.  AMMP  Refinement  Phases  using  Interrater  Reliability  and  Feasibility  Evaluation 
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Evaluation  of  AMMP  tasks  based  on  Task  Evaluation  Criteria:  Logistical  and  Practical  Considerations 

The  ultimate  goal  of  the  AMMP  development  effort  is  to  provide  military  clinicians  with  a  reliable  and 
valid  test  battery  that  is  clinically  implement  able.  Therefore,  logistical  and  practical  considerations  of  the 
individual  tasks  were  evaluated  (see  Tables  IA  &  IB). 

Test  Administration  Time 
Background 

Team  members  administering  the  AMMP  protocol  were  asked  to  time  the  total  administration  time  for 
each  of  the  tasks.  Total  administration  time  included  instruction  time  and  time  to  complete  the  task.  They  began 
recording  this  data  at  subject  ID  71.  In  total,  for  each  of  the  AMMP  tasks,  40  to  47  HC  and  23  to  30  mTBI 
participants  were  included  in  the  analysis. 

Method 

Subjects  were  excluded  if  they  had  not  completed  the  task,  since  their  task  times  are  not  reflective  of  the 
true  total  time.  Mean  and  median  times  are  reported  for  HC  and  mTBI  groups.  Tests  for  significant  differences 
between  HC  and  mTBI  included  the  t-Test  for  independent  groups  to  test  for  difference  of  means  if  the  data  met 
the  criteria  for  a  normal  distribution;  otherwise,  the  non-parametric  Mann-Whitney  U  test  for  independent  groups 
was  used  to  test  the  difference  in  medians.  A  two-sided  p-value  <0.05  was  required  for  significance. 

Results 

AMMP  task  administration  time  for  HCs  was  between  12.3  -  26.6  minutes  across  each  of  the  6  tasks  and 
between  12.1-30.1  minutes  to  administer  the  same  to  SM  with  mTBI.  For  CQ  Duty,  there  was  a  statistically 
significant  difference  in  administration  time  between  HC  and  SM  with  mTBI  (p  =  0.033).  Time  estimates  for  set 
up  and  take  down  were  quite  similar  across  tasks  (approximately  10  minutes  to  set  up  and  between  5-10  minutes 
to  take  down). 

Cost 

There  was  a  fairly  wide  range  of  supplies  and  equipment  costs  across  tasks,  ranging  from  $175  (CQ  Duty) 
to  $8,920  (Instrumented  Stand  and  Walk  -  Grid  Coordinates,  or  ISAW-Grid).  The  tasks  that  included  use  of 
inertial  sensor  systems  incurred  the  largest  equipment  costs  (Illinois  Agility  Test-Packing  List,  Run  Roll  Aim, 
ISAW-Grid).  Inertial  sensor  systems  are  becoming  more  readily  available  in  clinical  settings  and  may  not  be 
considered  cost  prohibitive. 
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Table  IA.  Logistical  and  practical  characteristics  for  AMMP  multitasks 


Parameter 

Description 

Charge  of  Quarters  Duty 

Run-Roll-Aim 

Patrol-Exertion 

HC  Mean 
(SD) 

N=44 

mTBI 

Mean  (SD) 
N=30 

HC  Mean 
(SD) 

N=40 

mTBI 

Mean  (SD) 
N=27 

HC  Mean 
(SD) 

N=47 

mTBI 

Mean  (SD) 
N=28 

Total  time 

Time  (minutes) 
from  beginning  of 
task  instructions  to 
end  of  task 
performance 

26.6  (6.3) 

30.1  (7.2) 

13.4  (4.1) 

14.0  (5.7) 

23.2  (3.2) 

24.5  (4.7) 

Set  up  and 
take  down 
time 

Total  time 

Approximately  10  minutes 
up  and  10  minutes  down 

Approximately  10  minutes 
and  5  minutes  down 

Approximately  10  minutes 
and  10  minutes  down 

Storage 

space 

Estimated  space 
needed  to  store 
materials,  supplies 
when  the  task  is 
not  set  up 

All  supplies  fit  the  5- 
drawer  storage  unit, 

24”(H)  x  13”(W)  x  12” 

(D) 

All  supplies  fit  in  a  small 
closet. 

All  supplies  fit  in  a  small 
closet. 

Cost 

Estimated  dollar 
amount  of 
materials  and 
equipment 

$180* 

$6,1272 

$1,3603 

1  PVC  pieces  ($50),  walkie  talkies  ($60),  office  supplies  ($15),  plastic,  5 -drawer  storage  unit  ($10),  stopwatch  ($15),  wall 
clock  ($30).  Does  not  include  tables  and  chairs  that  would  be  available  in  a  clinic  setting. 

2  Laptop  with  Microsoft  Office  Powerpoint  ($400),  2  traffic  cones  ($12),  slide  advancer  ($40),  simulated  weapon  ($300), 
stopwatch  ($15),  NexGen  inertial  sensor  system  (http://www.nexgenergo.com/ergonomics/ergoprods.html)  (2  sensors) 
($5,300),  short  focal  point  scope  (such  as  insect  or  bird  watching  scope  ($60)  mounted  on  simulated  weapon.  Does  not 
include  therapy  mats  that  would  be  available  in  a  clinic  setting. 

3  Laptop  (to  play  Patrol  video)  ($400),  audio -speakers  ($50),  large  video  monitor  ($350),  audio-recorder  ($50),  sports  type 
heart  rate  monitor  with  chest  strap  ($50),  helmet  ($100),  eye -protection  ($20),  aerobics  step  ($40),  simulated  weapon  ($300). 
Does  not  include  engineering  costs  and  components  for  creating  reaction  time  device  affixed  to  simulated  weapon. 
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Table  IB.  Logistical  and  practical  characteristics  of  AMMP  dual-tasks 


Parameter 

Description 

Illinois  Agility  Test- 
Packing  List 

ISAW  -  Grid 
Coordinates 

Load  Magazine  - 
Radio  Chatter 

HC  Mean 
(SD) 

N  =  46 

mTBI 

Mean 

(SD) 

N  =  23 

HC  Mean 
(SD) 

N  =  47 

mTBI 

Mean 

(SD) 

N  =  29 

HC  Mean 
(SD) 

N  =  43 

mTBI 

Mean 

(SD) 

N  =  29 

Total  time 

Time  (minutes)  from 
beginning  of  task 
instructions  to  end  of 
task  performance 

12.3 

(2.8) 

12.1  (2.7) 

16.2 

(5.3) 

17.8  (3.9) 

21.9  (4.0) 

22.1  (3.8) 

Set  up  and 
take  down 
time 

Total  time 

10  minutes  set  up,  5 
minutes  take  down 

10  minutes  set  up,  5 
minutes  take  down 

5  minutes  set  up,  5 
minutes  take  down 

Storage 

space 

Estimated  space 
needed  to  store 
materials  and  supplies 
when  the  task  is  not  set 
up 

All  supplies  fit  into  a 
small  closet 

All  supplies  fit  into  a 
small  cupboard 

All  supplies  fit  into  a 
small  cupboard 

Cost 

Estimated  dollar 
amount  of  materials 
and  equipment 

$5,4404 

$8,9205 

$4076 

4  Orange  traffic  cones  x  8  ($120),  tape  ($5),  stopwatch  ($15),  NextGen  sensor  system  (2  sensors)  ($5,300). 

5  Laptop  ($400),  stopwatch  ($15),  tape  ($5),  APDM  Inertial  Sensors  (Opal)  system  ($8,500)  (www.apdm.com/wearable- 
sensors) 

6  M-16  dummy  rounds  ($250),  M-16  magazines  ($50),  plastic  workshop  bins  ($12),  stopwatch  ($15),  iPod  (or  other  audio 
player)  ($50),  iPod  speaker  ($40) 
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AIM  2:  EVALUATE  INTER-RATER  RELIABILITY  FOR  EACH  OF  THE  DUAL-TASK  AND 
MULTITASKS  USING  HEALTHY  CONTROLS  AND  SM  WITH  DIAGNOSED  MTBI 

Introduction 

The  AMMP  team  focused  on  development  of  a  performance  based  assessment  that  targeted  known  mTBI 
vulnerabilities  to  be  used  to  help  inform  return  to  duty  (RTD)  decision-making  following  military  concussion. 

The  AMMP  had  to  meet  ecological  validity  standards  of  military  personnel,  including  commanders,  who  value 
the  real-world  use  of  the  assessment1.  Given  the  consequences  of  using  AMMP  metrics  to  contribute  to  duty 
readiness  decisions,  a  vital  first  step  in  the  AMMP  validation  process  was  to  determine  if  the  AMMP  tasks  could 
be  scored  reliably.  The  focus  of  Aim  2  of  this  protocol  was  to  evaluate  the  IRR  for  each  of  the  dual-tasks  and 
multitasks  using  HC  and  SM  with  diagnosed  mTBI. 

Method 

This  study  aim  involved  three  sequential  phases  which  used  IRR  findings,  informal  and  qualitative 
feedback  from  raters  and  subjects,  as  well  as  logistical  evaluation  by  the  test  developers  of  practical  properties  to 
drive  task  revision.  Specifically,  the  team  integrated  lessons  learned  during  testing  in  an  iterative  manner  over 
successive  phases  of  testing  to  improve  face  validity  of  the  tasks  and  the  quality  of  subject  performance  data. 

Research  design 

This  was  a  measurement  development  study  involving  test  construction  and  test  evaluation. 

Participants 
Phase  I 

Healthy  subjects  between  the  ages  of  18  and  42  were  recruited  by  convenience  sampling  from  both  HRV 
and  permanent  party  active  duty  service  members  from  the  USARIEM  in  Natick,  Massachusetts.  Inclusion  and 
exclusion  criteria  are  described  above.  In  Phase  la,  the  initial  9  AMMP  tasks  were  evaluated  on  20  HC  volunteers 
(11  males,  9  females,  mean  age  25.8  (+/-3.5)  with  revisions  made  that  required  re-evaluation  of  IRR  on  2  of  the 
tasks.  In  Phase  lb,  these  two  revised  tasks  (CQ  Duty  and  SALUTE-Exertion  task)  were  tested  on  12  USARIEM 
subjects  who  were  a  subset  of  the  subjects  tested  in  Phase  la. 

Phase  II 

A  total  of  12  SM  (11  male,  median  age  31,  range  21-42)  with  mTBI  were  recruited  from  the  clinical 
population  receiving  rehabilitation  services  at  the  WAMC  TBI  Clinic  for  persistent  symptoms  from  a  concussion 
occurring  from  2  weeks  to  2  years  (median  days  (range):  306  (71-470)  prior  to  the  AMMP  test  date.  Physical  and 
occupational  therapists  from  the  WAMC  TBI  Clinic  identified  potential  participants  who  met  the  eligibility 
criteria  and  provided  an  information  and  study  contact  form.  Participants  were  excluded  as  described  in  Phase  I. 

A  second  set  of  7  SM  (6  male)  were  recruited  from  this  same  population  for  additional  reliability  testing  that  was 
focused  primarily  on  revisions  to  the  SALUTE-exertion  task  scoring  and  instructions  to  subjects. 

Phase  III 

As  described  above,  HC  subjects  were  recruited  by  convenience  sampling  of  volunteers  from  Fort  Bragg 
Special  Operations  and  the  528  Sustainment  Brigade,  subjects  who  responded  to  recruitment  postings  placed  at 
fitness  centers  and  cafeterias  around  the  base,  as  well  as  volunteers  from  the  DVBIC  in-processing  briefings. 
These  DVBIC  briefings  occurred  on  an  almost  daily  basis  as  part  of  the  standard  informational  training  provided 
to  all  soldiers  in-processing  to  Fort  Bragg  after  transfer  from  other  duty  stations.  Subjects  with  mTBI  were 
recruited  from  the  clinical  population  receiving  rehabilitation  services  at  the  WAMC  TBI  Clinic  using  inclusion 
criteria  as  described  in  Phase  II  above.  A  total  of  26  subjects  were  involved  in  this  final  phase  of  IRR  evaluation, 
7  HC  (5  male,  median  (range)  age  34(20-42)  and  19  subjects  with  residual  mTBI  symptoms  (all  male,  median 
(range)  age  24  (19-40).  Median  days  since  most  recent  concussion  was  147  (range  63-632). 
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Analysis 


The  Krippendorf  Alpha  4  was  used  to  evaluate  IRR.  This  general  measure  can  be  used  regardless  of  the 
number  of  observers,  sample  size,  missing  data  and  type  of  measurement  (nominal,  ordinal,  interval,  or  ratio).  For 
both  interval  and  ratio  data  the  analysis  is  equivalent  to  the  intraclass  correlation  coefficient  (ICC)  for  two 
observers  and  is  extended  for  many  observers.  For  nominal  data,  analysis  for  two  observers  is  equivalent  to 
Scott’s  Pi.  Parallel  analyses  using  both  the  Krippendorf  and  Kappa  (2  observers)  have  produced  identical  results. 
The  code  was  integrated  into  both  SPSS  V18.0.  Bootstrapping  with  an  n=2000  was  used  to  produce  95% 
confidence  intervals.  In  some  cases  where  the  scorers  were  not  constant  the  SPSS  VI 8.0  ICC  analysis  assuming  a 
one  way  random  model  was  used  to  confirm  the  Krippendorff  result. 

Results 


IRR  findings  are  provided  in  Tables  IIA-IIF  with  the  dual-tasks  presented  first,  followed  by  the 
multitasks.  All  test  metrics  for  each  of  the  6  tasks  remaining  in  the  AMMP  battery  have  interrater  reliability 
above  0.90  for  dual-tasks  and  above  0.88  for  multitasks  except  for  metrics  on  errors  and  cues  which  were  then  not 
used  to  evaluate  between  group  differences. 
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Table  IIA.  Illinois  Agility-Packing  List  Interrater  Reliability 


Scoring  item 

USARIEM* 

n=20,  Healthy  Controls  (HC) 

Fort  Bragg/WAMC* 
n=12  SM  with  mTBI 

Fort  Bragg/WAMC# 
n=  23  (18  mTBI,  5  HC) 

(Metrics) 

Reliability,  ICC 

95%  Cl  (lower,  upper) 

Reliability,  ICC 

95%  Cl  (lower,  upper) 

Reliability,  ICC 

95%  Cl  (lower,  upper) 

Single  Task  Time 

0.98  (0.96-0.99) 

.82  (0.58-0.98) 

0.99  (0.98-0.99) 

Single  Task  Words  Correct 

0.89  (0.73-0.99) 

0.80  (0.69-0.90) 

1.0  (1-1) 

Single  Task  Word  Errors 

NA 

0.54  (0.12-0.83) 

1.0  (1-1) 

Dual-Task  No  Instruction:  Time 

0.96  (0.94-0.98) 

0.96  (0.93-0.98) 

0.99  (0.987-0.995) 

Dual-Task  NI:  Words  Correct 

0.93  (0.88-0.97) 

0.93  (0.86-0.99) 

0.99  (0.97-1) 

Dual-Task  NI:  Word  Errors 

0.07  (0-0.30) 

0.93  (0.87-0.97) 

0.99  (0.96-1) 

Dual-Task  NI:  Course  Errors 

NA 

NA 

1.0  (1-1) 

Dual-Task  COG:  Time 

0.90  (0.85-0.95) 

0.98  (0.97-0.99) 

0.99  (0. 989-. 997) 

Dual-Task  COG:  Words  Correct 

l.O(l.O-l.O) 

0.97  (0.92-1) 

1.0  (1-1) 

Dual-Task  COG:  Word  Errors 

0.10(0.0-0.36) 

0.74  (0.37-0.99) 

0.996  (0.987-1) 

Dual-Task  COG:  Course  Errors 

NA 

NA 

0.77  (0-1) 

Dual-Task  MOB:  Time 

0.96  (0.91-0.98) 

0.88  (0.80-0.95) 

0.98  (0.97-0.99) 

Dual-Task  MOB:  Words  Correct 

1.0  (1-1) 

1.0  (1-1) 

1.0  (1-1) 

Dual-Task  MOB:  Word  Errors 

0.09  (0-0.35) 

0.85  (0.64-1) 

0.86  (0.58-1) 

Dual-Task  MOB:  Course  Errors 

NA 

NA 

1.0  (1-1) 

COG:  Cognitive  priority;  “concentrate  on  remembering  the  words”,  NI: 
on  going  as  fast  as  you  can”  NA:  not  applicable  or  not  evaluated 
*  =  3  raters;  #  =  2  raters 

no  instruction  given,  MOB:  Mobility  priority;  “concentrate 
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Table  IIB.  Load  Magazine-Radio  Chatter  Interrater  Reliability 


Scoring  item 
(Metrics) 

USARIEM* 

n=20  Healthy  Controls  (HC) 

Fort  Bragg/WAMC* 
n=12  SM  with  mTBI 

Fort  Bragg/WAMC# 
n=  24  (18  mTBI,  6  HC) 

ICC  (95%  Cl) 

ICC  (95%  Cl) 

ICC  (95%  Cl) 

Rounds  loaded  single  &  dual® 

Not  evaluated 

Not  evaluated 

Not  evaluated 

Correct  Key  Word  Single 

0.99  (0.98-1) 

0.94  (0.88-0.99) 

0.997  (0.993-1) 

Distractor  Key  Word  Single 

0.93  (0.83-1) 

0.69  (0.38-0.92) 

0.995  (0.986-1) 

Correct  Key  Word  Dual 

0.98  (0.96-1) 

0.99  (0.97-1.0) 

0.978  (0.949-0.999) 

Distractor  Key  Word  Dual 

0.97  (0.95-0.99) 

0.50  (0.11-0.82) 

0.947  (0.869-1) 

*  =  3  raters;  #  =  2  raters;  @  = 
rater 

for  practical  reasons,  all  dummy  rounds  were  counted  one  time,  not  by  individual 

Table  IIC.  Instrumented  Stand  and  Walk-Grid  Coordinates  (ISAW-grid)  Interrater  Reliability 


Scoring  item 

USARIEM* 

n=20,  Healthy  Controls  (HC) 

Fort  Bragg/WAMC* 
n=10  SM  with  mTBI 

Fort  Bragg/WAMC  # 
n=  26  (19  mTBI,  7  HC) 

(Metrics) 

ICC  (95%  Cl) 

ICC  95%  Cl 

ICC  (95%  Cl) 

Walk  Time  1  Single 

0.99  (0.98-1), 

0.77  (0.64-0.86) 

0.97  (0.96-0.99) 

Walk  Time  2  Single 

0.97  (0.96-0.98) 

0.95  (0.92-0.98) 

0.95  (0.90-0.98) 

Walk  Time  3  Single 

0.98  (0.97-0.99) 

0.91  (0.85-0.96) 

0.98  (0.97-0.99) 

Walk  Time  1  Dual 

0.92  (0.85,0.97) 

0.89  (0.78-0.98) 

0.92  (0.86-0.97) 

Walk  Time  2  Dual 

0.98  (0.97,0.99) 

0.94  (0.89-0.96) 

0.95  (0.90-0.98) 

Walk  Time  3  Dual 

0.97  (0.95-0.98) 

0.81  (0.72-0.88) 

0.98  (0.97-0.99) 

Grid  Coord  Single 

0.56  (0.14-0.90) 

0.88  (0.78-0.97) 

0.97  (0.92-1) 

Grid  Coord  1  Dual 

0.94  (0.90-0.97) 

0.94  (0.85-1) 

0.98  (0.93-1) 

Grid  Coord  2  Dual 

0.99  (0.97-1) 

0.99  (0.99-1) 

0.999  (0.998-1) 

Grid  Coord  3  Dual 

0.92  (0.84-0.99) 

1.0  (1-1) 

0.997  (0.990-1) 

*  =  3  raters;  #  =  2  raters;  Coord  =  coordinates 
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Table  IIP.  Patrol-Exertion  Interrater  Reliability 


SALUTE-EXERTION  Version  1  (VI) 

SALUTE-EXERTION  Version  2  (V2) 

PATROL-EXERTION 

Scoring  item 

USARIEM* 
n=20  (Yl) 

USARIEM* 
n=12  (V2) 

Fort  Bragg/WAMC* 
n=7  SM  with  mTBI  (V2) 

Scoring  item 

Fort  Bragg/WAMC# 
n=  26  (19  mTBI,  7  HC) 

(Metrics) 

Reliability,  ICC 

95%  Cl  (lower,  upper) 

(Metrics) 

Reliability  ICC 

95%  Cl  (lower,  upper) 

Size 

0.85  (0.76-0.93) 

0.72  (0.53-0.87) 

3  triplets  disagreed1 

X.  Sum  of  IED  markers 

0.95  (0.91-0.98) 

Activity 

0.29  (0.0-0.60) 

0.77  (0.58-0.94) 

5  triplets  disagreed1 

Y.  Sum  of  post-test 
patrol  questions 

0.97  (0.94-1) 

Location 

0.80  (0.64-0.93) 

0.78  (0.56-0.95) 

1  triplet  disagreed1 

Z.  Sum  of  X  and  Y 

0.97  (0.95-0.99) 

Unit 

NA® 

0.14(0-0.57) 

3  triplets  disagreed1 

Vision  clarity  initial 

0.99  (0.97-1) 

Time 

0.57  (0.22-0.86) 

0.73  (0.44-0.96) 

1  triplet  disagreed1 

Vision  clarity  end 

0.99  (0.98-1) 

Equipment 

0.89  (0.79-0.92) 

0.81  (0.62-0.95) 

3  triplets  disagreed1 

RPE  initial 

0.98  (0.95-1) 

Scan  IED 
Markers 

NA 

0.90  (0.76-0.98) 

0.97  (0.94-0.99) 

RPE  end 

1.0  (1-1) 

Total  Score 

0.80  (0.66-0.91) 

0.79  (0.66-0.90) 

0.91  (0.84-0.96) 

@  =  In  the  initial  version,  the  report  was  described  as  a  SALTE  report  as  the  “Uniform  or  Unit”  component  of  the  report  was  not  consistently 
used,  based  on  advice  from  one  military  advisor  per  local  reporting  format. 

1  Due  to  insufficient  n  to  calculate  ICC,  the  number  of  rater  disagreements  is  reported 

RPE  =  Rate  of  Perceived  Exertion;  NA=not  applicable  or  not  evaluated 
*  =  3  raters;  #  =  2  raters;  Vl=Version  1,  V2=Version  2 
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Table  HE.  Charge  of  Quarters  (CQ)  Duty  Interrater  Reliability 


Scoring  item  n_ 

USARIEM  * 

=12  Healthy  Control  (HC) 

Fort  Bragg/WAMC* 
n=12  SM  with  mTBI 

Fort  Bragg/WAMC# 
n=  25  (19  mTBI,  6  HC) 

(Metrics) 

ICC  (95%  Cl) 

ICC  (95%  Cl) 

ICC  (95%  Cl) 

Task  performance 

0.94  (0.86-0.99) 

0.90  (0.84-0.95) 

0.88  (0.76-0.97) 

#  of  Rule  breaks 

0.64  (0.32-0.90) 

0.46  (0.0-0.79) 

0.91  (0.75-1) 

#  of  Visits 

0.98  (0.96-0.99) 

0.92  (0.80-0.99) 

0.98  (0.97-0.99) 

Total  time 

0.98  (0.96-0.99) 

0.99  (0.99-1.0) 

0.998  (0.994-1) 

*  =  3  raters;  #  =  2  raters 
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Table  IIF.  Run-Roll- Aim  Interrater  Reliability 


USARIEM* 

Fort  Bragg/WAMC* 

Fort  Bragg/WAMC  # 

n=20  Healthy  Controls  (HC) 

Scoring  item 

n=l  1  SM  mTBI® 

n=  26  (19  mTBI,  7  HC) 

Scoring  item 

ICC  (95%  Cl) 

(Metrics) 

Reliability,  ICC 

Reliability,  ICC 

95%  Cl  (lower,  upper) 

95%  Cl  (lower,  upper) 

Trial  1  incongruent 
numbers  correct 

0.996  (0.99-1) 

Trial  l-Time(secs) 

0.91  (0.80-0.99) 

0.999  (0.997-1) 

Trial  1  incongruent 
number  errors 

2  of  20  triplets 
disagreed 

Trial  1 -numbers 

correct 

0.54  (0.08-0.89) 

0.96  (0.91-1) 

Trial  2  congruent 
numbers  correct 

0.86  (0.65-1) 

Trial  2-Time  (secs) 

0.80  (0.57-0.97) 

0.999  (0.993-0.999) 

Trial  2  congruent 
number  errors 

2  of  20  disagreed 

Trial  2-numbers 

correct 

0.55  (0.0-0.93) 

0.93  (0.70-1) 

Trial  3  congruent 
numbers  correct 

0.57  (0.15-0.89) 

Trial  3-Time(secs) 

0.86  (0.67-0.98) 

0.995  (0.991-1) 

Trial  3  congruent 
number  errors 

2  of  20  disagreed 

Trial  3 -numbers 

correct 

0.72  (0.40-0.95) 

0.996  (0-1) 

Trial  4  incongruent 
numbers  correct 

0.50  (0.23-0.74) 

Trial  4-Time(secs) 

0.89  (0.75-0.98) 

0.999  (0.998-1) 

Trial  4  incongruent 
number  errors 

1  of  20  disagreed 

Trial  4-numbers 

correct 

0.99  (0.97-1.0) 

0.98  (0.96-1) 

ICC’s  for  individual  trials 

Total  errors  (all  trials) 

calculated,  Tl:  0.54,  T2:  0.13, 

0.64  (0.13-0.92) 

T3:  0.18,  T4:  0.85 

ICC’s  for  individual  trials 

Total  cues  (all  trials) 

calculated,  Tl:  0.71,  T2:  0.56, 

0.87  (0.66-1) 

_ T3:  0.37,  T4:  NA& _ 

Time  to  complete  the  Run-Roll-Aim  was  not  scored  by  all  raters  during  initial  testing. 

Errors  for  HC  were  recorded  as  #triplets  disagreed  as  the  range  of  errors  was  low,  ie.,  0-3  total; 

Prior  to  testing  mTBI,  revisions  were  made  to  score  sheets  and  instructions;  -Congruent  directional  Stroop  cue=  roll  direction  arrow 
and  R/L  letter  match,  incongruent  directional  Stroop  cue  =  roll  direction  arrow  and  R/L  letter  do  not  match: 

Correct  /  Errors  =  odd  or  even  numbers  viewed  through  scope  and  called  out:  &  No  cues  required — all  zeros. 

*=3  raters;  #=2  raters;  @  =  not  all  subjects  were  able  to  tolerate  completion  of  all  trials; 
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Limitations 


No  subjects  with  residual  symptoms  of  mTBI  were  less  than  2  months  post  most  recent  concussion  in  this 
study  due  to  recruitment  issues.  Scoring  of  and  responses  from  subjects  with  more  acute  symptoms  and  from 
populations  at  additional  installations  or  deployment  environments  may  result  in  a  requirement  for  further 
refinement  of  operational  definitions  of  task  metrics  including  expansion  of  acceptable  and  unacceptable  answers. 
All  raters  for  this  study  were  physical  and  occupational  therapists  with  a  minimum  of  6  years  of  experience  and  a 
knowledge  of  the  background  and  development  process  of  the  AMMP  battery.  This  may  have  contributed  to  a 
bias  in  scoring  some  or  all  of  the  AMMP  tasks.  Further  reliability  testing  with  novice  raters  who  did  not 
participate  in  the  development  of  this  assessment  will  contribute  to  clarity  on  the  amount  of  training  required  to 
achieve  adequate  IRR  for  a  clinical  metric. 

Conclusions 

Acceptable  levels  of  IRR  were  achieved  for  the  AMMP  dual-task  and  multitask  metrics  with  the 
application  of  a  deliberate  refinement  process  that  recognized  the  importance  of  measure  reliability  and 
feasibility.  In  this  early  stage  of  the  AMMP  validation  process,  we  have  chosen  to  deal  with  the  complexity  of  the 
multifaceted  metrics  that  are  used  in  dual-task  and  in  multitask  measures,  by  evaluating  IRR  for  each  separate  task 
metric.  As  the  AMMP  validation  process  proceeds,  we  aim  to  normalize  performance  across  AMMP  tasks, 
combining  individual  component  metrics  to  generate  a  composite  score  for  each  task  and  potentially  for  the 
complete  AMMP  battery.  Composite  scores  should  ease  interpretation  and  facilitate  decision  making  as 
demonstrated  with  other  batteries  described  in  the  rehabilitation  literature  [5,6j. 

The  AMMP  is  intended  for  use  in  combination  with  other  metrics  to  inform  RTD  decision-making  in  SM 
with  mTBI.  To  make  RTD  recommendations,  the  importance  of  IRR  in  a  metric  cannot  be  overstated  [7  8]. 

Kottner  et.  al.,  suggests  that  when  important  decisions  on  individuals  are  being  made  on  the  basis  of  an 
assessment  score,  IRR  values  should  be  0.90  or  0.95  [8].  Not  all  metrics  for  the  AMMP  tasks  met  this  stringent 
standard;  however,  following  the  iterative  process  in  the  AMMP  battery  development,  the  majority  of  the  ICC’s 
were  above  0.90  (Tables  IIA-IIF),  supporting  the  continuation  of  the  validation  process  for  the  component  tasks  in 
this  assessment  battery.  Those  metrics  that  did  not  meet  this  standard  were  typically  characterized  by  restricted 
value  ranges  which  can  significantly  reduce  ICC  values. 

The  consistency  of  scores  across  raters  is  fundamental  to  the  ability  to  use  the  findings  of  the  AMMP  to 
make  substantive  recommendations  regarding  readiness  to  RTD  following  concussion/mTBI.  This  research 
demonstrated  that  following  an  iterative  development  process,  individual  AMMP  tasks  appear  feasible,  and  have 
metrics  that  can  be  reliably  scored  by  experienced  rehabilitation  professionals.  Before  the  AMMP  is  used 
clinically  to  inform  RTD  decision-making,  further  evaluation  of  intra-rater,  novice  rater,  test-retest  reliability,  and 
additional  validation  studies  should  be  carried  out. 
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AIM  3:  DETERMINE  CORRELATION  BETWEEN  SCORES  ON  THE  NEUROBEHAVIORAL  AND 
SENSORIMOTOR  DOMAIN  TESTS  AND  SCORES  ON  AMMP  DUAL-  AND  MULTITASKS  IN 
HEALTHY  CONTROL  SM  AND  SM  WITH  MTBI. 


Introduction 

Design  of  the  AMMP  tasks  focused  on  attempting  to  test  known  vulnerabilities  associated  with 
neurobehavioral  and  sensorimotor  domains  affected  by  mTBI.  Table  IIIA  provides  hypothesized  correlations  of 
task  scores  and  these  vulnerabilities.  Correlation  of  tasks  with  neurobehavioral  tests  are  described  in  this  section. 
Correlations  were  not  hypothesized  for  the  physical  domains  thought  to  be  susceptible  to  concussion,  however,  in 
order  to  ensure  AMMP  task  inclusion  of  all  vulnerabilities,  the  physical  domains  are  included  in  the  task 
development  matrix. 


Table  IIIA.  Hypothesized  correlations  AMMP  Components  (gray)  and  Tests  of  Neurobehavioral  Domains 
(yellow) _ 


mTBI-related  task  challenges: 

Primary  •  Secondary  o 

mTBI  Susceptible  Domains 

Cognitive 

Sensorimotor 

Physical 

Executive 

function1 

Memory 

Attention 

Reaction 

time 

Eye  gaze 

tracking 

Scanning 

Vestibular 

Balance 

Exertion 

Bend-lift 

Manual 

Speed 

AMMP  MULTITASKS 

CQ  Duty 

• 

o 

o 

o 

o 

Run-Roll-Aim 

o 

o 

• 

• 

• 

o 

o 

Patrol-Exertion 

o 

o 

• 

• 

• 

o 

o 

• 

AMMP  DUAL  TASKS 

Illinois  Agility  Test-Packing  List 

• 

o 

o 

• 

o 

Load  Magazine-Radio  Chatter 

o 

• 

• 

Instrumented  Stand  &  Walk-Grid  coordinates 

• 

o 

o 

o 

NEUROBEHAVIORAL  TESTS  -  to  evaluate  validity  of  AMMP  components2 

Simple  Rx  Time:  Key  press,  decision-rule3 

• 

Neuropsychological  Assessment  Battery  (NAB) 
attention  module  1)  fwd/bkwd  digit  span  2) 
numbers  and  letters 

• 

• 

• 

Dynamic  Visual  Acuity:  Vertical  and  Horizontal  - 
head  still  vs.  head  moving  at  2Hz 

o 

• 

Tower  of  Hanoi  Planning  (ANAM4) 

• 

Comprehensive  Trail  Making  Test:  Connect 
numbers/letters  in  sequence  with  distractors 

• 

• 

• 

o 

WRAT  (Wide  Range  Achievement  Test)  Reading 

Reading  skills  and  IQ  estimate 

Test  of  Memory  Malingering 

Test-taking  effort 

Planning,  decision-making,  response  inhibition,  processing  speed 

2Neurobehavioral  tests  will  support  AMMP  construct  validity  but  not  included  in  the  final  AMMP  battery 

3Simple  Reaction  time  (. ANAM4 )  administered  at  the  beginning  and  end  of  test  session  as  a  marker  of  test  related 
fatigue.  (Bleiberg  et  al,  2007). 
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Methods 


Research  Design 

Data  source 

All  neurobehavioral  tests  were  administered  to  HC  at  the  time  they  were  tested  on  the  AMMP.  The  mTBI 
participants  were  tested  on  the  Tower  of  Hanoi  (TOH),  Wide  Range  Achievement  Test  (WRAT)  and  SRT  at  the  time 
they  were  tested.  All  other  test  results  for  this  group  were  extracted  from  their  medical  record. 

Analysis 

Pearson  Product-Moment  Correlation  Coefficient  was  used  to  evaluate  the  relationship  between  AMMP 
task  scores  and  neurobehavioral  test  performance.  Sensorimotor  measures  are  related  to  task  performance  and  are 
not  specifically  evaluated  here.  Task  scores  may  provide  insight  into  the  possible  effects  of  these  measures. 
Dynamic  Visual  Acuity  Test  (DVAT)  results  did  not  provide  evidence  of  deficiency  for  most  participants  where  a 
minimum  of  3  lines  loss  indicates  a  problem.  Only  three  HC  and  three  mTBI  met  this  criteria.  Therefore, 
correlation  analysis  was  not  carried  out. 

Tables  of  results 

The  tables  of  results  for  each  of  the  tasks  includes  the  following  rows:  SRT  (number  of  moves,  error 
percentile,  total  errors),  Neurobehavioral  Assessment  Battery  Numbers-Letters  Part  D  (NAB-D)  (raw  and  T- 
score),  Neurobehavioral  Assessment  Battery  Digit  Backward  (NAB_DB)  (raw  and  T-score),  TOH, 
Comprehensive  Trailmaking  Test  (CTMT)  Composite  Index,  and  WRAT  Reading  (raw  and  standardized  score). 
Columns  for  each  of  tasks  are  the  specific  task  scores. 

Correlation  coefficients  include  95%  confidence  intervals  and  a  p-value  as  a  measure  of  difference  from 
zero.  R  Statistics  (https ://www .r-proi ect . org/)  was  used  for  analysis. 

Participants 

Study  participants  included  all  persons  recruited  at  Fort  Bragg  for  the  evaluation  of  known  groups.  In 
addition,  subjects  recruited  at  Fort  Bragg  for  evaluation  of  IRR  were  included  in  tasks  that  did  not  change  and 
exhibited  high  reliability.  This  included  Load  Magazine-Radio  Chatter,  ISAW-Grid  and  number  of  visits  and  total 
time  scores  of  the  CQ  Duty.  Participants  not  completing  a  task  were  excluded. 

Results 


Refer  to  Tables  IIIB  through  IIIG  for  specific  results.  Most  significant  correlations  were  small  at  less  than 

0.4.  This  was  expected  given  the  complexity  of  the  tasks  and  likelihood  of  multiple  influences  on  performance.  A 

minimum  correlation  of  0.3  was  selected  for  evaluation  of  hypothesized  relationships  with  task  metrics. 

Highlighted  items  exhibit  absolute  correlation  values  >03. 

CQ  Duty  (Table  IIIB) 

•  Demonstrated  a  small  correlation  with  TOH  as  a  measure  of  executive  function,  as  hypothesized,  for  task 
metrics  of  Total  Performance  Score  and  #  visits. 

•  Demonstrated  a  small  correlation  with  NAB  Numbers-Letters,  as  hypothesized,  as  a  measure  of  executive 
function  and  memory/attention  for  task  metric  #  visits. 

•  Demonstrated  a  small  correlation  with  CTMT  as  a  measure  of  executive  function  and  attention,  as 
hypothesized  for  task  metrics  of  total  time. 

•  Demonstrated  a  small  correlation  with  SRT  (baseline  and  end  of  testing),  not  hypothesized,  for  task  metric 
task  performance.  This  may  be  related  to  level  of  fatigue. 

Run-Roll-Aim  (Table  IIIC) 
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•  Demonstrated  a  small  correlation  with  NAB  Numbers-Letters  as  a  measure  of  memory/attention,  as 
hypothesized,  for  task  metrics  total  time  for  each  of  the  trials  1  to  4  as  well  as  aggregate  time  (sum  of  trials  1 
to  4). 

•  Demonstrated  a  small  correlation  with  NAB_DB  as  a  measure  of  memory/attention,  as  hypothesized  for  task 
metrics  total  time  on  trial  4. 

•  Demonstrated  a  small  correlation  with  CTMT  as  a  measure  of  executive  function  and  attention,  as 
hypothesized,  for  task  metrics  total  time  trials  2  and  4  and  aggregate  time. 

•  Did  not  correlate  with  SRT  as  hypothesized. 

Patrol-Exertion  (Table  HID) 

•  Did  not  demonstrate  correlation  of  >0.3  with  any  of  the  neurocognitive  measures. 

•  Demonstrated  a  small  correlation  with  SRT  at  baseline  and  moderate  correlation  with  SRT  at  end  of  testing, 
as  hypothesized,  for  task  metrics  related  to  vision  clarity  and  reaction  time  at  both  the  beginning  and  end  of 
testing. 

Illinois  Agility-Packing  List  (Table  HIE) 

•  Demonstrated  a  small  correlation  with  TOH  as  a  measure  of  executive  function,  not  hypothesized,  for  task 
metrics  dual  test  time  with  emphasis  on  remembering  the  words  and  dual-task  time  with  emphasis  on  mobility 
as  well  as  aggregate  time. 

•  Demonstrated  a  small  correlation  with  NAB_DB  as  a  measure  of  memory/attention,  as  hypothesized,  for  task 
metrics  #  words  correct  under  no  instruction  condition. 

•  Demonstrated  a  small  correlation  with  WRAT  as  a  measure  of  IQ,  not  hypothesized,  for  #  words  remembered 
under  all  conditions. 

•  Demonstrated  a  small  correlation  with  SRT,  as  hypothesized,  for  task  metrics  time  for  each  of  the  trials  with 
the  exception  of  the  trial  emphasizing  the  cognitive  aspect  of  the  task  to  remember  the  words. 

Load  Magazine-Radio  Chatter  (Table  IIIF) 

•  Demonstrated  a  small  correlation  with  NAB  Numbers-Letters  as  a  measure  of  executive  function  and 
attention,  as  hypothesized,  for  task  metric  errors  under  dual-task  condition. 

•  Demonstrated  a  small  correlation  of  CTMT  as  a  measure  of  executive  function  and  attention,  as  hypothesized, 
for  task  metrics  #  correct  and  errors  under  dual-task  condition. 

•  Demonstrated  a  small  correlation  with  WRAT  as  a  measure  of  IQ,  not  hypothesized,  for  task  metrics  #  correct 
and  errors  under  dual-task  condition. 

ISAW-Grid  (Table  IIIG) 

•  Demonstrated  a  small  correlation  with  NAB_DB  as  a  measure  of  memory/attention,  as  hypothesized,  for  task 
metric  of  #  grid  coordinates  remembered  under  single  task  condition. 

•  Demonstrated  a  small  correlation  with  WRAT  as  a  measure  of  IQ,  not  hypothesized,  for  task  metric  #  grid 
coordinates  remembered  under  single  task  condition. 

Conclusions 

Hypothesized  correlations  with  neuro-behavioral  measures,  although  small,  were  demonstrated  for  most 

tasks  with  the  exception  of  the  Patrol-Exertion  task.  Correlations  were  small,  possibly  due  to  complexity  of  the 

tasks  with  multiple  factors  affecting  performance.  In  addition,  the  mTBI  group  were  not  tested  at  the  time  of 

performing  AMMP  for  NAB  and  CTMT  tests,  which  may  have  been  a  confounder  given  lag  times. 
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Table  IIIB.  CQ  Duty.  CQ  Duty  task  results.  Results  are  presented  as:  Pearson  correlation  (95%  Cl),  number  of  subjects  used  to  calculate  the  correlation, 
and  p-value.  An  asterisk  next  to  a  p-value  denotes  statistical  significance  at  level  0.05.  Includes  only  subjects  who  completed  the  task. 


Complete  sample 

A.  Task  Performance 

B.  Specific  Rule 

D.  Total  Visits  to 

Total  Score 

Breaks  Total 

C.  #  Rule  Breaks  Total 

Complete  Task 

Total  Time  (Mins) 

Tower  of  Hanoi:  # 

-0.3  (-0.48,  -0.09) 

0.14  (-0.08,  0.35) 

0.25  (0.03,  0.44) 

0.36  (0.17,0.53) 

0.21  (0.01,0.4) 

moves 

n=83,  p=0.006* 

n=83  p=0.203 

n=83,  p=0.025* 

n=92,  p<0.001* 

n=92,  p=0.041* 

Tower  of  Hanoi:  Error 

-0.19  (-0.39,  0.03) 

0.14  (-0.07,  0.35) 

0.12  (-0.1,  0.33) 

0.36  (0.16,  0.53) 

0.09  (-0.12,  0.3) 

percentile 

n=83  p=0.085 

n=83  p=0.193 

n=83  p=0.278 

n=83,p=0.001* 

n=83  p=0.399 

Tower  Hanoi  Error: 

-0.32  (-0.5, -0.12) 

0.12  (-0.09,  0.33) 

0.15  (-0.07,  0.35) 

0.45  (0.26,  0.6) 

0.12  (-0.1,  0.33) 

Total 

n=83,  p=0.003* 

n=83  p=0.266 

n=83  p=0. 1 86 

n=83,p<0.001* 

n=83  p=0.28 

NAB  Numbers-letters: 

0.23  (0.01,0.44) 

-0.05  (-0.28,0.17) 

-0.14  (-0.35,  0.09) 

-0.3  (-0.47,  -0.09) 

-0.2  (-0.4,  0) 

Part  D  raw 

n=76,  p=0.041* 

n=76  p=0.639 

n=76  p=0.227 

n=89,  p=0.005* 

n=89  p=0.054 

NAB  Numbers-Letters: 

0.2  (-0.02,  0.41) 

-0.02  (-0.25,  0.2) 

-0.09  (-0.31,0.14) 

-0.3  (-0.48,  -0.1) 

-0.18  (-0.37,  0.03) 

Part  D  T-score 

n=77  p=0.076 

n=77  p=0.832 

n=77  p=0.44 

n=92,  p=0.004* 

n=92  p=0.089 

NAB  Digit:  Backward 

0.19  (-0.04,  0.4) 

-0.18  (-0.39,  0.05) 

-0.09  (-0.31,0.14) 

-0.05  (-0.25,0.17) 

-0.16  (-0.36,  0.05) 

raw 

n=74  p=0.103 

n=74  p=0.133 

n=74  p=0.46 

n=87  p=0.674 

n=87  p=0.144 

NAB  Digit:  Backward 

0.2  (-0.03,  0.41) 

-0.18  (-0.39,  0.05) 

-0.1  (-0.32,  0.13) 

-0.01  (-0.22,  0.19) 

-0.15  (-0.34,  0.06) 

T-score 

n=75  p=0.087 

n=75  p=0.115 

n=75  p=0.393 

n=90  p=0.904 

n=90  p=0. 169 

CTMT :  Composite 

0.28  (0.06,  0.47) 

-0.04  (-0.27,0.18) 

-0.16  (-0.37,  0.07) 

-0.21  (-0.4,  -0.01) 

-0.4  (-0.56,  -0.21) 

Index 

n=76,  p=0.015* 

n=76  p=0.713 

n=76  p=0.171 

n=91,p=0.041* 

n=91,  p<0.001* 

WRAT :  Raw  score 

0.29  (0.08,  0.48) 

-0.14  (-0.35,  0.08) 

-0.13  (-0.34,  0.09) 

-0.25  (-0.43,  -0.06) 

-0.22  (-0.4,  -0.03) 

n=83,  p=0.007* 

n=83  p=0.207 

n=83  p=0.243 

n=102,  p=0.01* 

n=102,  p=0.024* 

WRAT:  Standardized 

0.29  (0.08,  0.48) 

-0.15  (-0.35,0.07) 

-0.16  (-0.37,  0.05) 

-0.26  (-0.43,  -0.06) 

-0.22  (-0.4,  -0.03) 

score 

n=83,  p=0.007* 

n=83  p=0. 1 86 

n=83  p=0. 1 36 

n=101,p=0.01* 

n=101,p=0.024* 

SRT:  Baseline 

-0.33  (-0.51,-0.12) 

0  (-0.21,  0.22) 

0.07  (-0.15,0.28) 

-0.04  (-0.24,  0.17) 

-0.02  (-0.22,  0.19) 

n=83,  p=0.003* 

n=83  p=0.968 

n=83  p=0.536 

n=91  p=0.709 

n=91  p=0.859 

SRT:  End  of  testing 

-0.41  (-0.57,  -0.21) 

0.04  (-0.18,0.25) 

0.04  (-0.18,0.25) 

-0.02  (-0.23,0.19) 

-0.04  (-0.25,0.17) 

n=81,p<0.001* 

n=81  p=0.749 

n=81  p=0.737 

n=87  p=0.862 

n=87  p=0.707 
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Table  IIIC(l).  Run-Roll- Aim  1.  Run-Roll- Aim  task  results.  Results  are  presented  as:  Pearson  correlation  (95%  Cl),  number  of  subjects  used  to  calculate 
the  correlation,  and  p-value.  An  asterisk  next  to  a  p-value  denotes  statistical  significance  at  level  0.05.  Includes  only  subjects  who  completed  the  task. 


Complete  sample 

Trial  1-  Correct 

Trial  2  Correct 

Trial  3  Correct 

Triall  Total  Time 

responses  / 14 

Trial2  Total  Time 

Responses  / 14 

Trial3  Total  Time 

Responses  / 14 

Tower  of  Hanoi:  # 

0.13  (-0.09,  0.34) 

-0.11  (-0.32,  0.11) 

0.06  (-0.17,  0.27) 

-0.08  (-0.3,0.14) 

0  (-0.22,  0.22) 

0.06  (-0.16,  0.27) 

moves 

n=80  p=0.241 

n=80  p=0.342 

n=80  p=0.62 1 

n=80  p=0.465 

n=79  p=0.978 

n=80  p=0.61 

Tower  of  Hanoi:  Error 

0.17  (-0.05,  0.37) 

0.1  (-0.12,  0.31) 

0.07  (-0.15,0.28) 

-0.05  (-0.27,0.17) 

0.19  (-0.03,  0.39) 

-0.03  (-0.25,0.19) 

percentile 

n=80  p=0. 1 38 

n=80  p=0.371 

n=80  p=0.55 

n=80  p=0.643 

n=79  p=0.097 

n=80  p=0.783 

Tower  Hanoi  Error: 

0.17  (-0.05,  0.38) 

0.08  (-0.14,  0.3) 

0.09  (-0.13,0.31) 

-0.1  (-0.31,0.12) 

0.18  (-0.05,  0.38) 

0.01  (-0.21,0.23) 

Total 

n=80  p=0.123 

n=80  p=0.465 

n=80  p=0.405 

n=80  p=0.386 

n=79  p=0.117 

n=80  p=0.915 

NAB  Numbers-letters: 

-0.31  (-0.5,  -0.08 j 

0.1  (-0.13,0.32) 

-0.4  (-0.58,  -0.19) 

-0.13  (-0.35,0.1) 

-0.37  (-0.56,  -0.16) 

-0.03  (-0.26,  0.2) 

Part  D  raw 

n=73,  p=0.009* 

n=73  p=0.396 

n=73,p<0.001* 

n=73  p=0.258 

n=72,  p=0.001* 

n=73  p=0.773 

NAB  Numbers- 

-0.26  (-0.46,  -0.03) 

0.1  (-0.13,0.32) 

-0.33  (-0.52,-0.11) 

-0.08  (-0.3,0.15) 

-0.32  (-0.51,-0.09) 

-0.03  (-0.26,  0.2) 

Letters:  Part  D  T-score 

n=74,  p=0.026* 

n=74  p=0.401 

n=74,  p=0.004* 

n=74  p=0.489 

n=73,  p=0.007* 

n=74  p=0.792 

NAB  Digit:  Backward 

-0.15  (-0.37,0.09) 

-0.1  (-0.33,0.13) 

-0.13  (-0.35,  0.11) 

0.07  (-0.16,  0.3) 

-0.11  (-0.33,0.13) 

-0.11  (-0.34,  0.12) 

raw 

n=71  p=0.215 

n=71  p=0.394 

n=71  p=0.288 

n=71  p=0.554 

n=70  p=0.378 

n=71  p=0.349 

NAB  Digit:  Backward 

-0.18  (-0.4,  0.05) 

-0.1  (-0.32,  0.13) 

-0.17  (-0.39,  0.06) 

0.1  (-0.14,  0.32) 

-0.12  (-0.34,  0.12) 

0.01  (-0.23,  0.24) 

T-score 

n=72  p=0.129 

n=72  p=0.404 

n=72  p=0.153 

n=72  p=0.416 

n=71  p=0.333 

n=72  p=0.958 

CTMT :  Composite 

-0.28  (-0.48,  -0.05) 

0.15  (-0.08,0.37) 

-0.31  (-0.5, -0.08) 

-0.2  (-0.41,0.03) 

-0.23  (-0.44,  0) 

-0.06  (-0.28,0.18) 

Index 

n=73,  p=0.018* 

n=73  p=0.201 

n=73,  p=0.008* 

n=73  p=0.086 

n=72,  p=0.049* 

n=73  p=0.641 

WRAT :  Raw  score 

-0.15  (-0.36,  0.07) 

0.09  (-0.13,0.31) 

-0.14  (-0.35,  0.08) 

0.12  (-0.1,  0.33) 

-0.17  (-0.38,  0.05) 

-0.11  (-0.32,  0.11) 

n=80  p=0.191 

n=80  p=0.417 

n=80  p=0.201 

n=80  p=0.287 

n=79  p=0.137 

n=80  p=0.32 

WRAT:  Standardized 

-0.19  (-0.39,  0.03) 

0.11  (-0.11,0.32) 

-0.15  (-0.36,  0.07) 

0.09  (-0.13,0.3) 

-0.13  (-0.34,  0.1) 

-0.22  (-0.42,  0) 

score 

n=80  p=0.09 

n=80  p =0.323 

n=80  p=0.178 

n=80  p=0.438 

n=79  p=0.259 

n=80,  p=0.047* 

SRT:  Baseline 

0.06  (-0.16,  0.28) 

0.03  (-0.19,  0.24) 

0.2  (-0.02,  0.41) 

0.11  (-0.11,0.32) 

0.11  (-0.12,  0.32) 

0.03  (-0.19,  0.25) 

n=80  p=0.587 

n=80  p=0.817 

n=80  p=0.069 

n=80  p=0.324 

n=79  p=0.347 

n=80  p=0.779 

SRT:  End  of  testing 

0.07  (-0.15,0.29) 

0.07  (-0.16,  0.29) 

0.29  (0.07,  0.48) 

-0.03  (-0.25,0.19) 

0.19  (-0.03,  0.4) 

0.04  (-0.18,0.26) 

n=78  p=0.528 

n=78  p =0.55 5 

n=78,  p=0.011* 

n=78  p=0.781 

n=77  p=0.092 

n=78  p=0.706 
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Table  IIIC(2).  Run-Roll-Aim  2.  Run-Roll- Aim  task  results.  Results  are  presented  as:  Pearson  correlation  (95%  Cl),  number  of  subjects  used  to  calculate 
the  correlation,  and  p-value.  An  asterisk  next  to  a  p-value  denotes  statistical  significance  at  level  0.05.  Includes  subjects  who  completed  the  task. 

Complete  sample 


Trail4  Total  Time 

Trial  4  Correct 
Responses  / 14 

RRA  Aggregate 
Correct 

RRA_Aggregate 

Error 

RRA  Aggregate  Time 

Tower  of  Hanoi:  #  moves 

0.17  (-0.05,  0.38) 

-0.12  (-0.33,  0.1) 

-0.11  (-0.32,  0.11) 

0.11  (-0.12,  0.32) 

0.11  (-0.11,0.32) 

n=80  p=0. 1 3 

n=80  p=0.279 

n=80  p =0.326 

n=80  p=0.35 1 

n=80  p=0.317 

Tower  of  Hanoi:  Error 

0.13  (-0.1,  0.34) 

0.13  (-0.1,  0.34) 

0.09  (-0.14,  0.3) 

-0.08  (-0.29,  0.15) 

0.16  (-0.06,  0.37) 

percentile 

n=80  p=0.264 

n=80  p=0.265 

n=80  p=0.452 

n=80  p=0.501 

n=80  p=0. 146 

Tower  Hanoi  Error:  Total 

0.16  (-0.06,  0.37) 

0.07  (-0.15,0.28) 

0.05  (-0.17,0.27) 

-0.06  (-0.28,0.16) 

0.18  (-0.04,  0.39) 

n=80  p=0. 1 55 

n=80  p=0.547 

n=80  p=0.636 

n=80  p=0.591 

n=80  p=0.104 

NAB  Numbers-letters: 

-0.45  (-0.62,  -0.25) 

0.05  (-0.18,0.28) 

0.01  (-0.22,  0.24) 

-0.06  (-0.28,0.18) 

-0.44  (-0.61,-0.23) 

Part  D  raw 

n=73,  p<0.001* 

n=73  p=0.658 

n=73  p=0.956 

n=73  p=0.631 

n=73,  p<0.001* 

NAB  Numbers-Letters: 

-0.42  (-0.59,  -0.21) 

0.02  (-0.21,0.25) 

0.01  (-0.22,  0.23) 

-0.06  (-0.28,0.17) 

-0.37  (-0.55,-0.16) 

Part  D  T-score 

n=74,  p<0.001* 

n=74  p=0.853 

n=74  p=0.957 

n=74  p=0.632 

n=74,  p=0.001* 

NAB  Digit:  Backward 

-0.3  (-0.5,  -0.07) 

0.14  (-0.1,  0.36) 

-0.01  (-0.24,  0.23) 

-0.03  (-0.26,  0.21) 

-0.19  (-0.4,  0.05) 

raw 

n=71,p=0.011* 

n=71  p=0.24 

n=71  p=0.956 

n=71  p=0.829 

n=71  p=0.118 

NAB  Digit:  Backward  T- 

-0.35  (-0.54,  -0.13) 

0.15  (-0.08,  0.37) 

0.09  (-0.15,0.31) 

-0.12  (-0.34,  0.11) 

-0.23  (-0.44,  0) 

score 

n=72,  p=0.002* 

n=72  p=0.208 

n=72  p=0.468 

n=72  p=0.314 

n=72  p=0.055 

CTMT :  Composite  Index 

-0.41  (-0.58,  -0.2) 

0.18  (-0.05,  0.4) 

0.07  (-0.16,  0.3) 

-0.11  (-0.33,0.12) 

-0.35  (-0.54,  -0.13) 

n=73,p<0.001* 

n=73  p=0.122 

n=73  p=0.528 

n=73  p=0.344 

n=73,  p=0.002* 

WRAT :  Raw  score 

-0.16  (-0.37,  0.06) 

-0.11  (-0.33,0.11) 

-0.07  (-0.28,0.15) 

0.05  (-0.17,0.27) 

-0.19  (-0.4,  0.03) 

n=80  p=0. 1 56 

n=80  p=0.3 1 3 

n=80  p=0.548 

n=80  p=0.663 

n=80  p=0.087 

WRAT:  Standardized 

-0.16  (-0.37,  0.06) 

-0.13  (-0.34,  0.09) 

-0.15  (-0.36,  0.07) 

0.12  (-0.11,  0.33) 

-0.2  (-0.4,  0.02) 

score 

n=80  p=0. 158 

n=80  p =0.24 6 

n=80  p=0.184 

n=80  p=0.309 

n=80  p=0.078 

SRT:  Baseline 

0.06  (-0.16,  0.28) 

0.03  (-0.19,  0.24) 

0.2  (-0.02,  0.41) 

0.11  (-0.11,0.32) 

0.11  (-0.12,  0.32) 

n=80  p=0.587 

n=80  p=0.817 

n=80  p=0.069 

n=80  p=0.324 

n=79  p=0.347 

SRT:  End  of  testing 

0.07  (-0.15,0.29) 

0.07  (-0.16,  0.29) 

0.29  (0.07,  0.48) 

-0.03  (-0.25,0.19) 

0.19  (-0.03,  0.4) 

n=78  p=0.528 

n=78  p =0.555 

n=78,  p=0.011* 

n=78  p=0.781 

n=77  p=0.092 
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Table  IIID(l).  Patrol-Exertion  1.  Patrol-Exertion  task  results.  Results  are  presented  as:  Pearson  correlation  (95%  Cl),  number  of  subjects  used 
to  calculate  the  correlation,  and  p-value.  An  asterisk  next  to  a  p-value  denotes  statistical  significance  at  level  0.05.  Includes  only 
subjects  who  completed  the  task. _ 


Complete  sample 

Z.  Total  points  (X  + 
Y)/24 

How  hard  are  you 
working  STANDING 

How  hard  are  you 
working  PRE¬ 
VIDEO 

How  hard  are  you 
working  END 

VIDEO 

Tower  of  Hanoi:  #  moves 

-0.12  (-0.33,  0.09) 

0.01  (-0.22,  0.25) 

0  (-0.21,  0.22) 

0.07  (-0.15,0.28) 

n=84  p=0.262 

n=70  p=0.914 

n=82  p=0.984 

n=84  p=0.546 

Tower  of  Hanoi:  Error 

-0.29  (-0.48,  -0.09) 

0.23  (-0.01,0.44) 

0.1  (-0.12,  0.31) 

0.09  (-0.13,0.3) 

percentile 

n=84,  p=0.007* 

n=70  p=0.061 

n=82  p=0.351 

n=84  p=0.412 

Tower  Hanoi  Error:  Total 

-0.24  (-0.43,  -0.02) 

0.2  (-0.04,  0.41) 

0.06  (-0.16,  0.28) 

0.15  (-0.07,  0.35) 

n=84,  p=0.03* 

n=70  p=0.101 

n=82  p=0.572 

n=84  p=0.173 

NAB  Numbers-letters: 

0.21  (-0.01,0.42) 

-0.01  (-0.26,  0.23) 

0.02  (-0.21,0.25) 

-0.1  (-0.32,  0.13) 

Part  D  raw 

n=77  p=0.064 

n=64  p=0.91 

n=75  p=0.871 

n=77  p=0.391 

NAB  Numbers-Letters: 

0.2  (-0.02,  0.41) 

-0.05  (-0.29,  0.2) 

0.01  (-0.21,0.24) 

-0.1  (-0.32,  0.12) 

Part  D  T-score 

n=78  p=0.077 

n=65  p=0.689 

n=76  p=0.901 

n=78  p=0.374 

NAB  Digit:  Backward 

-0.14  (-0.36,  0.09) 

-0.16  (-0.39,  0.09) 

0.02  (-0.21,0.25) 

-0.07  (-0.29,  0.16) 

raw 

n=75  p=0.226 

n=63  p=0.22 

n=73  p=0.87 

n=75  p=0.572 

NAB  Digit:  Backward  T- 

-0.13  (-0.35,0.1) 

-0.2  (-0.43,  0.05) 

0.05  (-0.19,  0.27) 

-0.05  (-0.28,0.17) 

score 

n=76  p=0.258 

n=64  p=0. 108 

n=74  p=0.702 

n=76  p=0.648 

CTMT :  Composite  Index 

0.2  (-0.03,  0.41) 

-0.23  (-0.45,  0.02) 

-0.11  (-0.33,0.12) 

-0.14  (-0.35,  0.09) 

n=77  p=0.082 

n=64  p=0.072 

n=75  p=0.341 

n=77  p=0.234 

WRAT :  Raw  score 

0.26  (0.05,  0.45) 

-0.07  (-0.3,0.17) 

0.07  (-0.15,0.28) 

-0.1  (-0.31,0.12) 

n=84,  p=0.016* 

n=70  p=0.575 

n=82  p=0.532 

n=84  p=0.373 

WRAT:  Standardized 

0.23  (0.01,0.42) 

-0.01  (-0.24,  0.23) 

0.15  (-0.07,  0.36) 

-0.07  (-0.28,0.14) 

score 

n=84,  p=0.039* 

n=70  p=0.944 

n=82  p=0.173 

n=84  p=0.498 

SRT:  Baseline 

-0.17  (-0.37,0.04) 

0.08  (-0.16,  0.31) 

-0.02  (-0.24,  0.2) 

0.11  (-0.11,0.31) 

n=84  p=0.118 

n=70  p=0.526 

n=82  p=0.865 

n=84  p=0.333 

SRT:  End  of  testing 

0.04  (-0.18,0.26) 

0.11  (-0.13,0.34) 

0.1  (-0.13,0.31) 

0.23  (0.01,0.42) 

n=81  p=0.73 

n=68  p=0.371 

n=79  p=0.392 

n=81,p=0.043* 
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Table  IIID(2).  Patrol-Exertion  2.  Patrol-Exertion  task  results.  Results  are  presented  as:  Pearson  correlation  (95%  Cl),  number  of  subjects  used  to 
calculate  the  correlation,  and  p- value.  An  asterisk  next  to  a  p-value  denotes  statistical  significance  at  level  0.05.  Includes  only  subjects  who  completed  the 
task. _ 

Complete 

sample 


Tower  of  Hanoi:  # 
moves 

Tower  of  Hanoi: 
Error  percentile 
Tower  Hanoi 
Error  Total 
NAB  Numbers - 
letters:  Part  D  raw 
NAB  Numbers - 
Letters:  Part  DT- 
score 

NAB  Digit: 
Backward  raw 
NAB  Digit: 
Backward  T- score 
CTMT: 

Composite  Index 

WRAT:  Raw 

score 

WRAT: 

Standardized 

score 

SRT:  Baseline 


SRT:  End  of 
testing 


Work  Change 
Stand  to  End 

0.1  (-0.14,  0.33) 
n=70  p=0.412 
-0.06  (-0.29,  0.18) 
n=70  p=0.643 
0.05  (-0.19,  0.28) 
n=70  p=0.683 
-0.06  (-0.31,0.18) 
n=64  p=0.611 
-0.06  (-0.3,  0.18) 
n=65  p=0.61 

0.05  (-0.2,  0.3) 
n=63  p=0.689 
0.1  (-0.15,  0.34) 
n=64  p=0.444 
0  (-0.24,  0.25) 
n=64  p=0.985 
0.05  (-0.18,  0.28) 
n=70  p=0.665 
0.02  (-0.22,  0.25) 
n=70  p=0.869 

0.11  (-0.13,  0.34) 
n=70  p=0.367 

0.13  (-0.12,  0.35) 
n=68  p=0.306 


Work  Change 
Pre  to  End _ 

0.07  (-0.15,  0.29) 
n=82  p=0.51 
0.01  (-0.2,  0.23) 
n=82  p=0.911 
0.11  (-0.11,0.32) 
n=82  p=0.34 
-0.14  (-0.36,  0.09) 
n=75  p=0.2 1 8 
-0.14  (-0.36,  0.09) 
n=76  p=0.2 19 

-0.08  (-0.31,0.15) 
n=73  p=0.493 
-0.09  (-0.31,0.14) 
n=74  p=0.447 
-0.04  (-0.27,  0.19) 
n=75  p=0.714 
-0.16  (-0.36,  0.06) 
n=82  p=0.158 
-0.21  (-0.41,0.01) 
n=82  p=0.062 

0.13  (-0.09,  0.34) 
n=82  p=0.233 

0.17  (-0.05,  0.38) 
n=79  p=0.134 


Vision  Clarity 
STANDING 

0.06  (-0.17,  0.3) 
n=70  p=0.595 
0.1  (-0.13,0.33) 
n=70  p=0.39 
0.12  (-0.12,  0.34) 
n=70  p=0.327 
-0.09  (-0.33,0.16) 
n=64  p=0.473 
-0.13  (-0.36,  0.12) 
n=65  p=0.32 

-0.1  (-0.34,  0.15) 
n=63  p=0.425 
-0.09  (-0.33,0.16) 
n=64  p=0.461 
-0.22  (-0.44,  0.03) 
n=64  p=0.079 
0.05  (-0.19,  0.28) 
n=70  p=0.698 
0.06  (-0.18,  0.29) 
n=70  p=0.623 

0.24  (0,  0.45) 
n=70,  p=0.049* 

0.21  (-0.03,  0.42) 
n=68  p=0.092 


Vision  Clarity 
PRE-VIDEO 

0.03  (-0.18,  0.25) 
n=82  p=0.761 
0.04  (-0.18,  0.25) 
n=82  p=0.73 
0.07  (-0.15,0.28) 
n=82  p=0.542 
-0.08  (-0.3,0.15) 
n=75  p=0.518 
-0.11  (-0.32,  0.12) 
n=76  p=0.356 

-0.11  (-0.33,0.12) 
n=73  p=0.344 
-0.09  (-0.32,  0.14) 
n=74  p=0.425 
-0.21  (-0.42,  0.02) 
n=75  p=0.07 
0.1  (-0.12,  0.31) 
n=82  p=0.369 
0.11  (-0.11,0.32) 
n=82  p=0.339 

0.33  (0.12,  0.51) 
n=82,  p=0.003* 

0.37  (0.16,  0.55) 
n=79,p=0.001* 


Vision  Clarity 
END  VIDEO 

0.15  (-0.07,  0.35) 
n=84  p=0.17 
0.18  (-0.03,  0.38) 
n=84  p=0.099 
0.25  (0.04,  0.44) 
n=84,  p=0.021* 
-0.05  (-0.27,0.17) 
n=77  p=0.644 
-0.06  (-0.28,0.16) 
n=78  p=0.576 

-0.13  (-0.35,0.1) 
n=75  p=0.253 
-0.12  (-0.34,  0.11) 
n=76  p=0.307 
-0.24  (-0.44,  -0.02) 
n=77,  p=0.033* 
0.1  (-0.12,  0.31) 
n=84  p=0.364 
0.09  (-0.12,  0.3) 
n=84  p=0.392 

0.35  (0.14,  0.52) 
n=84,  p=0.001* 

0.58(0.41,0.71) 

n=81,p<0.001* 


Vision  Change 
Stand  to  End 

0.2  (-0.03,  0.42) 
n=70  p=0.094 
0.19  (-0.05,  0.41) 
n=70  p=0. 1 1 6 
0.29  (0.06,  0.49) 
n=70,  p=0.015* 
-0.05  (-0.29,  0.2) 
n=64  p=0.678 
-0.06  (-0.3,0.19) 
n=65  p=0.649 

-0.17  (-0.4,  0.08) 
n=63  p=0.184 
-0.16  (-0.39,  0.09) 
n=64  p=0. 199 
-0.28  (-0.49,  -0.04) 
n=64,  p=0.024* 
0.07  (-0.17,  0.3) 
n=70  p=0.563 
0.07  (-0.17,  0.3) 
n=70  p=0.592 

0.35  (0.13,0.54) 
n=70,  p=0.003* 

0.65  (0.49,  0.77) 
n=68,  p<0.001* 


Vision  Change 
_Pre  to  End _ 

0.24  (0.03,  0.44) 
n=82,  p=0.028* 
0.28  (0.07,  0.47) 
n=82,  p=0.01* 

0.37  (0.17,0.55) 
n=82,  p=0.001* 
-0.05  (-0.27,0.18) 
n=75  p=0.667 
-0.04  (-0.26,  0.19) 
n=76  p=0.75 

-0.13  (-0.35,0.11) 
n=73  p=0.29 
-0.12  (-0.34,  0.11) 
n=74  p=0.32 
-0.24  (-0.44,  -0.01) 
n=75,  p=0.038* 
0.06  (-0.16,  0.28) 
n=82  p=0.569 
0.04  (-0.17,  0.26) 
n=82  p=0.696 

0.32  (0.11,0.5) 
n=82,  p=0.004* 

0.65  (0.5,  0.76) 
n=79,  p<0.001* 
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Table  IIID(3).  Patrol-Exertion  3.  Patrol-Exertion  task  results.  Results  are  presented  as:  Pearson  correlation  (95%  Cl),  number  of  subjects  used  to 
calculate  the  correlation,  and  p- value.  An  asterisk  next  to  a  p-value  denotes  statistical  significance  at  level  0.05.  Includes  only  subjects  who  completed  the 
task. 


Complete  sample 

Patrol_Reaction  Baseline 
Standing 

Patrol  Reactionjbaseline- 
Stepping 

PatrolReactBEGIN 

PatrolReactEND 

Tower  of  Hanoi:  #  moves 

0.14  (-0.07,  0.35) 
n=84  p=0. 1 89 

-0.18  (-0.38,  0.04) 
n=84  p=0.111 

-0.02  (-0.24,  0.2) 
n=80  p=0.85 

0.02  (-0.2,  0.23) 
n=8 1  p=0.882 

Tower  of  Hanoi:  Error 

-0.22  (-0.42,  -0.01) 

-0.08  (-0.29,  0.14) 

-0.08  (-0.29,  0.15) 

0.12  (-0.1,  0.33) 

percentile 

n=84,  p=0.041* 

n=84  p=0.463 

n=80  p=0.498 

n=81  p=0.296 

Tower  Hanoi  Error  Total 

-0.15  (-0.35,0.06) 
n=84  p=0.169 

-0.1  (-0.31,0.12) 
n=84  p=0.36 

-0.07  (-0.29,  0.15) 
n=80  p=0.535 

0.08  (-0.14,  0.3) 
n=81  p=0.463 

NAB  Numbers-letters:  Part  D 

-0.11  (-0.33,0.12) 

-0.05  (-0.27,0.18) 

-0.19  (-0.4,  0.05) 

-0.16  (-0.38,  0.07) 

raw 

n=77  p=0.335 

n=77  p=0.663 

n=73  p=0.113 

n=74  p=0.161 

NAB  Numbers-Letters:  Part  D 

-0.11  (-0.33,0.11) 

-0.03  (-0.25,  0.2) 

-0.11  (-0.33,0.12) 

-0.1  (-0.32,  0.13) 

T-score 

n=78  p=0.329 

n=78  p=0.807 

n=74  p=0.362 

n=75  p=0.374 

NAB  Digit:  Backward  raw 

-0.19  (-0.4,  0.04) 
n=75  p=0.097 

-0.14  (-0.35,  0.09) 
n=75  p=0.247 

-0.09  (-0.31,0.15) 
n=71  p=0.48 

-0.28  (-0.48,  -0.05) 
n=72,  p=0.019* 

NAB  Digit:  Backward  T-score 

-0.22  (-0.43,  0) 
n=76  p=0.053 

-0.18  (-0.39,  0.05) 
n=76  p=0.12 

-0.08  (-0.3,0.16) 
n=72  p=0.522 

-0.21  (-0.42,  0.02) 
n=73  p=0.069 

CTMT :  Composite  Index 

-0.15  (-0.36,  0.08) 
n=77  p=0.194 

-0.1  (-0.32,  0.13) 
n=77  p=0.393 

-0.02  (-0.25,  0.21) 
n=73  p=0.864 

-0.2  (-0.41,0.03) 
n=74  p=0.093 

WRAT :  Raw  score 

-0.01  (-0.22,  0.2) 
n=84  p=0.921 

0.1  (-0.12,  0.31) 
n=84  p =0.364 

-0.21  (-0.41,0.01) 
n=80  p=0.056 

-0.1  (-0.31,0.12) 
n=8 1  p=0.358 

WRAT :  Standardized  score 

-0.03  (-0.24,  0.19) 
n=84  p=0.789 

0.07  (-0.15,0.28) 
n=84  p=0.538 

-0.25  (-0.44,  -0.03) 
n=80,  p=0.026* 

-0.13  (-0.34,  0.09) 
n=81  p=0.254 

SRT:  Baseline 

0.11  (-0.1,0.32) 
n=84  p=0.303 

0.18  (-0.04,  0.38) 
n=84  p=0.1 

0.18  (-0.04,  0.39) 
n=80  p=0.104 

0.26  (0.05,  0.45) 
n=81,p=0.018* 

SRT:  End  of  testing 

0.1  (-0.12,  0.31) 
n=81  p=0.37 

0.18  (-0.04,  0.39) 
n=81  p=0.102 

0.31  (0.09,  0.5) 
n=77,  p=0.006* 

0.45  (0.26,  0.61) 
n=78,  p<0.001* 
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Table  IIIE(l).  Agility  1.  Illinois  Agility-Packing  List  task  results.  Results  are  presented  as:  Pearson  correlation  (95%  Cl),  number  of  subjects  used  to 
calculate  the  correlation,  and  p- value.  An  asterisk  next  to  a  p-value  denotes  statistical  significance  at  level  0.05.  Includes  only  subjects  who  completed  the 
task. 


Complete  sample _ 

Dual  WORD 


Single  Task 
Agility  Time 

Single  Task 
Words  Correct 

Single  Task 
Word  Errors 

DNI  Task 

Words  Correct 

DNI  Task  Test 
Time 

DNI  Total 
Word  Errors 

Task  Words 
Correct 

Dual  WORD 
Task  Test  Time 

Tower  of  Hanoi: 

0.08 

-0.13 

0.16 

-0.09 

0.15 

-0.04 

-0.05 

0.23 

#  moves 

(-0.13,0.29) 

(-0.33,  0.08) 

(-0.05,  0.36) 

(-0.29,  0.13) 

(-0.07,  0.35) 

(-0.25,0.18) 

(-0.26,  0.16) 

(0.02,  0.42) 

n=86  p=0.467 

n=86  p=0.235 

n=86  p =0.141 

n=86  p=0.43 

n=86  p=0.173 

n=86  p=0.734 

n=86  p=0.646 

n=86,  p=0.034* 

Tower  of  Hanoi: 

0.13 

-0.13 

0.21 

-0.14 

0.19 

0.04 

-0.1 

0.25 

Error  percentile 

(-0.1,0.34) 

(-0.34,  0.1) 

(-0.02,  0.41) 

(-0.35,  0.09) 

(-0.03,  0.4) 

(-0.19,  0.26) 

(-0.32,  0.13) 

(0.03,  0.45) 

n=77  p=0.263 

n=77  p=0.27 

n=77  p=0.074 

n=77  p=0.229 

n=77  p=0.09 

n=77  p=0.733 

n=77  p=0.388 

n=77,  p=0.029* 

Tower  Hanoi 

0.16 

-0.12 

0.19 

-0.11 

0.27 

0.02 

-0.06 

0.3 

Error  Total 

(-0.07,  0.37) 

(-0.34,  0.11) 

(-0.04,  0.39) 

(-0.32,  0.12) 

(0.05,  0.47) 

(-0.21,0.24) 

(-0.28,0.17) 

(0.08,  0.49) 

n=77  p=0.176 

n=77  p=0.293 

n=77  p=0.104 

n=77  p=0.35 

n=77, 

n=77  p=0.892 

n=77  p=0.605 

n=77,  p=0.009* 

p=0.017* 

NAB  Numbers- 

-0.03 

0.09 

-0.17 

-0.03 

-0.12 

0.15 

0.11 

-0.14 

letters:  Part  D 

(-0.25,0.18) 

(-0.12,  0.3) 

(-0.37,  0.04) 

(-0.25,0.18) 

(-0.32,  0.1) 

(-0.06,  0.36) 

(-0.1,0.32) 

(-0.34,  0.08) 

raw 

n=84  p=0.757 

n=84  p=0.39 

n=84  p=0.117 

n=84  p=0.762 

n=84  p=0.288 

n=84  p=0. 16 

n=84  p=0.301 

n=84  p=0.207 

NAB  Numbers- 

0.02 

0.11 

-0.18 

-0.02 

-0.07 

0.16 

0.1 

-0.1 

Letters:  Part  DT- 

(-0.2,  0.23) 

(-0.11,0.31) 

(-0.37,  0.04) 

(-0.23,  0.2) 

(-0.27,0.15) 

(-0.05,  0.36) 

(-0.11,0.31) 

(-0.3,0.12) 

score 

n=86  p=0.886 

n=86  p=0.322 

n=86  p=0. 103 

n=86  p=0.874 

n=86  p=0.545 

n=86  p=0. 1 33 

n=86  p=0.356 

n=86  p=0.378 

NAB  Digit: 

-0.15 

0.17 

-0.09  (-0.3, 

0.32 

0.01 

-0.18 

0.08 

0.01 

Backward  raw 

(-0.36,  0.07) 

(-0.05,  0.37) 

0.13) 

(0.11,0.51) 

(-0.21,0.23) 

(-0.38,  0.04) 

(-0.14,  0.29) 

(-0.2,  0.23) 

n=82  p=0.176 

n=82  p=0.13 

n=82  p=0.445 

n=82,  p=0.003* 

n=82  p=0.921 

n=82  p=0.116 

n=82  p=0.491 

n=82  p=0.895 

NAB  Digit: 

-0.18 

0.12 

-0.02 

0.24 

-0.02 

-0.11 

0.06 

-0.01 

Backward  T- 

(-0.38,  0.04) 

(-0.1,0.33) 

(-0.23,  0.2) 

(0.03,  0.43) 

(-0.23,0.19) 

(-0.32,  0.11) 

(-0.16,  0.27) 

(-0.22,  0.21) 

score 

n=84  p=0.107 

n=84  p=0.276 

n=84  p=0.883 

n=84,  p=0.026* 

n=84  p=0.854 

n=84  p=0.328 

n=84  p=0.61 

n=84  p=0.936 

CTMT: 

-0.14 

0.09 

-0.04 

-0.11 

-0.15 

0.2 

0.23 

-0.08 

Composite  Index 

(-0.34,  0.07) 

(-0.12,  0.3) 

(-0.25,0.17) 

(-0.31,0.11) 

(-0.35,  0.07) 

(-0.02,  0.39) 

(0.01,  0.42) 

(-0.29,  0.14) 

n=85  p=0.197 

n=85  p=0.388 

n=85  p=0.703 

n=85  p=0.33 

n=85  p=0.176 

n=85  p=0.072 

n=85,  p=0.037* 

n=85  p=0.466 

WRAT:  Raw 

0.01 

0.32 

0.36 

0.17 

-0.09 

0.4 

0.11 

score 

(-0.19,  0.21) 

(0.12,  0.49) 

(-0.48,  -0.11) 

(0.17,0.52) 

(-0.04,  0.36) 

(-0.29,  0.11) 

(0.22,  0.56) 

(-0.09,  0.31) 

n=95  p=0.911 

n=95,  p=0.002* 

n=95,p=0.003* 

n=95,  p<0.001* 

n=95  p=0.105 

n=95  p=0.38 

n=95,  p<0.001* 

n=95  p=0.268 
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Complete  sample _ 

Dual  WORD 


Single  Task 
Agility  Time 

Single  Task 
Words  Correct 

Single  Task 
Word  Errors 

DNI  Task 

Words  Correct 

DNI  Task  Test 
Time 

DNI  Total 
Word  Errors 

Task  Words 
Correct 

Dual  WORD 
Task  Test  Time 

WRAT: 

0.01 

0.34 

-0.32  (-0.49,  - 

0.34 

0.16 

-0.06 

0.42 

0.1 

Standardized 

(-0.19,  0.21) 

(0.15,0.51) 

0.13)  n=95, 

(0.15,0.51) 

(-0.04,  0.35) 

(-0.26,  0.14) 

(0.23,  0.57) 

(-0.1,  0.3) 

score 

n=95  p=0.946 

n=95,  p=0.001* 

p=0.001* 

n=95,  p=0.001* 

n=95  p=0.122 

n=95  p=0.56 

n=95,  p<0.001* 

n=95  p=0.333 

SRT:  Baseline 

0.31 

0.03 

-0.04 

0.11 

0.31 

-0.1 

-0.11 

0.27 

(0.1,0.49) 

(-0.19,  0.24) 

(-0.26,  0.17) 

(-0.11,0.31) 

(0.11,0.49) 

(-0.3,0.12) 

(-0.31,0.11) 

(0.06,  0.45) 

n=85, 

n=85  p=0.811 

n=85  p =0.68 5 

n=85  p =0.3 37 

n=85, 

n=85  p=0.374 

n=85  p=0.33 

n=85,  p=0.014* 

p=0.004* 

p=0.004* 

SRT:  End  of 

0.33 

0.04 

-0.1 

0 

0.33 

-0.02 

-0.04 

0.27 

testing 

(0.12,  0.52) 

(-0.19,  0.25) 

(-0.31,0.13) 

(-0.22,  0.22) 

(0.12,  0.51) 

(-0.24,  0.2) 

(-0.26,  0.18) 

(0.05,  0.46) 

n=80, 

n=80  p=0.755 

n=80  p=0.399 

n=80  p=0.974 

n=80, 

n=80  p=0.875 

n=80  p=0.732 

n=80,  p=0.017* 

p=0.003* 

p=0.003* 
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Table  IIIE(2).  Agility  2.  Illinois  Agility-  Packing  List  task  results.  Results  are  presented  as:  Pearson  correlation  (95%  Cl),  number  of  subjects  used  to 
calculate  the  correlation,  and  p- value.  An  asterisk  next  to  a  p-value  denotes  statistical  significance  at  level  0.05.  Includes  only  subjects  who  completed  the 
task. _ 

Complete  sample _ 

Dual  AGILITY  Dual  AGILITY  Aggregate 

Dual  WORD  Task  Words  Dual  AGILITY  Total  WORD  TOTAL_WORD  Aggregate  Aggregate  Total 

Total  Errors  Correct  Test  Time  Errors  S  Correct  Total Time  WORD  Errors 


Tower  of  Hanoi:  # 
moves 

Tower  of  Hanoi: 
Error  percentile 
Tower  Hanoi  Error 
Total 

NAB  Numbers- 
letters:  Part  D  raw 
NAB  Numbers- 
Letters:  Part  DT- 
score 

NAB  Digit: 
Backward  raw 
NAB  Digit: 
Backward  T-score 
CTMT :  Composite 
Index 

WRAT :  Raw  score 
WRAT: 

Standardized  score 
SRT:  Baseline 

SRT:  End  of  testing 


-0.03  (-0.24,  0.19) 

n=86  p=0.808 

0.02  (-0.21,0.24) 
n=77  p=0.882 
-0.02  (-0.24,  0.21) 
n=77  p=0.888 
0.01  (-0.21,0.22) 
n=84  p=0.957 
0.04  (-0.17,0.25) 
n=86  p=0.724 

0.04  (-0.18,0.25) 
n=82  p=0.732 
0.05  (-0.17,0.26) 
n=84  p=0.67 
-0.16  (-0.36,  0.06) 
n=85  p=0.147 
-0.22  (-0.4,  -0.02) 
n=95,  p=0.031* 
-0.22  (-0.4,  -0.02) 
n=95,  p=0.035* 
0.14  (-0.08,  0.34) 
n=85  p=0.216 
0.05  (-0.17,0.26) 
n=80  p=0.678 


-0.1  (-0.3,0.12) 
n=86  p=0.376 
-0.07  (-0.29,  0.16) 
n=77  p=0.57 
0.01  (-0.21,0.24) 
n=77  p=0.916 
0.01  (-0.2,  0.23) 
n=84  p=0.914 
0.02  (-0.19,  0.23) 
n=86  p=0.863 

0.23  (0.01,0.43) 
n=82,  p=0.038* 
0.26  (0.04,  0.45) 
n=84,  p=0.019* 
0.13  (-0.08,  0.34) 
n=85  p=0.231 
0.36  (0.17,0.52) 
n=95,  p<0.001* 
0.42  (0.24,  0.57) 
n=95,  p<0.001* 
-0.07  (-0.28,0.14) 
n=85  p=0.502 
0.03  (-0.19,  0.25) 

n=80  p=0.801 


0.13  (-0.09,  0.33) 
n=86  p =0.24 8 
0.25  (0.02,  0.45) 
n=77,  p=0.031* 
0.31  (0.1,  0.5) 
n=77,  p=0.006* 
-0.15  (-0.35,0.07) 
n=84  p=0.175 
-0.1  (-0.31,0.11) 
n=86  p=0.34 

0.04  (-0.18,0.25) 
n=82  p=0.726 
-0.02  (-0.23,  0.2) 
n=84  p=0.865 
-0.14  (-0.34,  0.07) 
n=85  p=0.197 
0.17  (-0.03,  0.36) 
n=95  p=0.104 
0.15  (-0.05,  0.34) 
n=95  p=0.14 
0.37  (0.17,0.54) 
n=85,  p=0.001* 
0.45  (0.26,  0.61) 
n=80,  pcO.OOl* 


-0.06  (-0.27,0.16) 
n=85  p=0.609 
-0.06  (-0.28,0.16) 
n=76  p=0.59 
-0.14  (-0.35,  0.09) 
n=76  p=0.227 
0.1  (-0.12,  0.31) 
n=83  p=0.354 
0.11  (-0.11,0.31) 
n=85  p=0.33 

-0.1  (-0.31,0.12) 
n=8 1  p=0.388 
-0.15  (-0.36,  0.06) 
n=83  p=0. 165 
-0.01  (-0.22,  0.21) 
n=84  p=0.937 
-0.09  (-0.29,  0.11) 
n=94  p=0.364 
-0.15  (-0.34,  0.06) 
n=94  p=0.158 
0.1  (-0.12,  0.31) 
n=84  p=0.372 
-0.04  (-0.26,  0.18) 
n=79  p=0.703 


-0.1  (-0.31,0.11) 
n=86  p=0.355 
-0.13  (-0.35,0.1) 
n=77  p=0.253 
-0.07  (-0.29,  0.16) 
n=77  p=0.561 
0.04  (-0.17,0.26) 
n=84  p=0.695 
0.05  (-0.17,0.26) 

n=86  p=0.668 

0.27  (0.06,  0.46) 
n=82,  p=0.015* 
0.24  (0.02,  0.43) 
n=84,  p=0.03* 
0.12  (-0.1,  0.32) 
n=85  p=0.292 
0.49  (0.32,  0.63) 
n=95,  p<0.001* 
0.51  (0.34,  0.65) 
n=95,  p<0.001* 
-0.03  (-0.25,0.18) 
n=85  p=0.751 
-0.01  (-0.23,  0.21) 
n=80  p=0.96 


0.18  (-0.03,  0.38) 
n=86  p=0.092 
0.24  (0.02,  0.44) 
n=77,  p=0.033* 
0.31  (0.09,  0.5) 
n=77,  p=0.006* 
-0.14  (-0.35,  0.07) 
n=84  p=0. 196 
-0.09  (-0.3,0.12) 
n=86  p=0.392 

0.02  (-0.2,  0.24) 
n=82  p=0.844 
-0.02  (-0.23,  0.2) 
n=84  p=0.882 
-0.13  (-0.33,0.09) 
n=85  p=0.244 
0.16  (-0.05,  0.35) 
n=95  p=0.132 
0.14  (-0.06,  0.33) 
n=95  p=0.167 
0.33  (0.12,  0.51) 
n=85,  p=0.002* 
0.36  (0.16,  0.54) 
n=80,  p=0.001* 


-0.05  (-0.26,  0.17) 
n=86  p=0.677 
0  (-0.22,  0.22) 
n=77  p=0.996 
-0.06  (-0.28,0.17) 
n=77  p=0.618 
0.12  (-0.1,  0.32) 
n=84  p=0.293 
0.13  (-0.08,  0.34) 
n=86  p=0.217 

-0.1  (-0.31,0.12) 
n=82  p=0.386 
-0.09  (-0.3,0.12) 
n=84  p=0.395 
-0.01  (-0.22,  0.21) 
n=85  p=0.949 
-0.18  (-0.37,  0.02) 
n=95  p=0.079 
-0.19  (-0.38,  0.01) 
n=95  p=0.064 
0.07  (-0.15,0.28) 
n=85  p=0.533 
0  (-0.22,  0.22) 
n=80  p=0.975 
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Table  IIIF.  Load  Magazine-Radio  Chatter.  Correlation  results  are  presented  as:  Pearson  correlation  (95%  Cl),  number  of  subjects  used  to  calculate  the 
correlation,  and  p-value.  An  asterisk  next  to  a  p-value  denotes  statistical  significance  at  level  0.05.  Includes  only  subjects  who  completed  the  task. 


Complete  sample 

3.  Correct  Single/ 

RadioChatErrors 

RadioChatErrors 

RadioChatErrorsDual 

RadioChatCorrect 

9 

5.  Correct  Dual/  9 

Single 

Dual 

minus single 

Sing-CorrectDual 

Tower  of  Hanoi:  # 

-0.07  (-0.27,0.14) 

-0.14  (-0.33,  0.07) 

0.09  (-0.12,  0.29) 

0.17  (-0.03,  0.36) 

0.04  (-0.17,0.24) 

0.04  (-0.17,0.24) 

moves 

n=92  p=0.508 

n=92  p=0.197 

n=92  p=0.385 

n=92  p=0.102 

n=92  p=0.714 

n=92  p=0.71 1 

Tower  of  Hanoi: 

-0.14  (-0.34,  0.08) 

-0.19  (-0.39,  0.02) 

0.15  (-0.07,0.36) 

0.2  (-0.02,  0.4) 

-0.01  (-0.22,  0.21) 

0.02  (-0.2,  0.23) 

Error  percentile 

n=82  p=0.22 

n=82  p=0.082 

n=82  p=0.169 

n=82  p=0.073 

n=82  p=0.959 

n=82  p=0.887 

Tower  Hanoi  Error 

-0.15  (-0.35,0.07) 

-0.18  (-0.38,  0.04) 

0.17  (-0.05,  0.37) 

0.18  (-0.04,  0.38) 

-0.03  (-0.25,0.19) 

-0.01  (-0.22,  0.21) 

Total 

n=82  p=0. 1 88 

n=82  p=0.111 

n=82  p=0.134 

n=82  p=0.101 

n=82  p=0.768 

n=82  p=0.953 

NAB  Numbers- 

0.14  (-0.07,  0.34) 

0.24  (0.03,  0.42) 

-0.13  (-0.33,  0.08) 

-0.27  (-0.45,  -0.07) 

-0.08  (-0.28,0.13) 

-0.04  (-0.25,0.17) 

letters:  Part  D  raw 

n=90  p=0.179 

n=90,  p=0.025* 

n=90  p=0.22 

n=90,  p=0.01* 

n=90  p=0.479 

n=90  p=0.689 

NAB  Numbers- 

0.16  (-0.05,  0.35) 

0.28  (0.08,  0.46) 

-0.13  (-0.33,  0.07) 

-0.31  (-0.48,-0.11) 

-0.1  (-0.3,0.11) 

-0.06  (-0.26,  0.14) 

Letters:  Part  D  T- 

n=93  p=0.136 

n=93,  p=0.007* 

n=93  p=0.204 

n=93,  p=0.003* 

n=93  p=0.34 

n=93  p=0.542 

score 

NAB  Digit: 

0.11  (-0.1,0.31) 

0.16  (-0.05,  0.36) 

-0.02  (-0.23,0.19) 

-0.14  (-0.34,  0.07) 

-0.08  (-0.29,  0.13) 

-0.01  (-0.22,  0.2) 

Backward  raw 

n=88  p=0.305 

n=88p=0.134 

n=88  p=0.845 

n=88  p=0.197 

n=88  p=0.432 

n=88  p=0.894 

NAB  Digit: 

0.16  (-0.05,  0.35) 

0.19  (-0.01,  0.38) 

-0.1  (-0.3,0.11) 

-0.19  (-0.38,  0.02) 

-0.04  (-0.25,0.16) 

0  (-0.2,  0.21) 

Backward  T-score 

n=91  p=0.141 

n=91  p=0.065 

n=91  p=0.365 

n=91  p=0.074 

n=91  p=0.673 

n=91  p=0.986 

CTMT :  Composite 

0.15  (-0.06,  0.34) 

0.35  (0.15,0.52) 

-0.15  (-0.34,  0.06) 

-0.41  (-0.57,  -0.22) 

-0.16  (-0.35,  0.05) 

-0.12  (-0.32,  0.08) 

Index 

n=92  p=0.167 

n=92,  p=0.001* 

n=92  p=0.157 

n=92,  p<0.001* 

n=92  p=0.133 

n=92  p=0.237 

WRAT :  Raw  score 

0.15  (-0.05,  0.33) 

0.35  (0.17,0.51) 

-0.1  (-0.29,  0.1) 

-0.35  (-0.51,-0.17) 

-0.18  (-0.36,  0.02) 

-0.14  (-0.32,  0.06) 

n=102  p=0.139 

n=102,  p<0.001* 

n=102  p=0.318 

n=102,  p<0.001 

* 

n=102  p=0.077 

n=102  p=0.17 

WRAT:  Standardized 

0.13  (-0.06,  0.32) 

0.33  (0.15,0.5) 

-0.1  (-0.29,  0.1) 

-0.33  (-0.49,  -0.14) 

-0.16  (-0.34,  0.04) 

-0.14  (-0.32,  0.06) 

score 

n=101  p=0.184 

n=101,  p=0.001* 

n=101  p=0.328 

n=101,  p=0.001 

* 

n=101  p=0.109 

n=101  p=0.172 

SRT:  Baseline 

-0.18  (-0.37,  0.03) 

-0.19  (-0.38,  0.02) 

0.18  (-0.03,  0.37) 

0.29  (0.09,  0.47) 

0.05  (-0.16,  0.25) 

-0.02  (-0.23,0.19) 

n=90  p=0.099 

n=90  p=0.072 

n=90  p=0.093 

n=90,  p=0.005* 

n=90  p=0.659 

n=90  p=0.847 

SRT:  End  of  testing 

-0.12  (-0.32,  0.1) 

-0.06  (-0.26,  0.16) 

0.15  (-0.07,  0.35) 

0.13  (-0.09,  0.33) 

-0.05  (-0.26,  0.16) 

-0.07  (-0.28,0.14) 

n=86  p=0.292 

n=86  p =0.60 9 

n=86  p=().  1 8 

n=86  p=0.241 

n=86  p=0.648 

n=86  p=0.523 
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Table  IIIG.  ISAW-Grid.  ISAW-Grid  task  results  are  presented  as:  Pearson  correlation  (95%  Cl),  number  of  subjects  used  to  calculate  the  correlation,  and 
p-value.  An  asterisk  next  to  a  p-value  denotes  statistical  significance  at  level  0.05.  Includes  only  subjects  who  completed  the  task. 


Complete  sample 


Single  Task  Median  Time 
(secs) 

Single  Task  Cognitive:  # 
Grid  Coordinates  Correct 

DualTask  Median  Time 

(secs) 

Dual- task  Cognitive:  #  Grid 
Coordinates  Correct 

Tower  of  Hanoi:  #  moves 

0.03  (-0.18,0.23) 

-0.19  (-0.38,  0.02) 

0.04  (-0.17,0.24) 

-0.09  (-0.29,  0.11) 

n=91  p=0.805 

n=91  p=0.073 

n=93  p=0.715 

n=93  p=0.378 

Tower  of  Hanoi:  Error 

0.02  (-0.19,  0.24) 

-0.21  (-0.41,0) 

0.08  (-0.13,0.29) 

0.07  (-0.14,  0.28) 

percentile 

n=83  p=0.836 

n=83  p=0.053 

n=84  p=0.444 

n=84  p=0.506 

Tower  Hanoi  Error  Total 

0.03  (-0.19,  0.25) 

-0.24  (-0.44,  -0.03) 

0.1  (-0.12,  0.31) 

0.07  (-0.15,0.28) 

n=83  p=0.777 

n=83,  p=0.027* 

n=84  p=0.373 

n=84  p=0.518 

NAB  Numbers-letters:  Part  D 

-0.09  (-0.29,  0.13) 

0.13  (-0.08,  0.34) 

-0.05  (-0.25,0.16) 

-0.15  (-0.35,0.06) 

raw 

n=86  p=0.426 

n=86  p=0.215 

n=90  p=0.641 

n=90  p=0.162 

NAB  Numbers-Letters:  Part  D 

-0.06  (-0.27,0.15) 

0.13  (-0.08,0.33) 

-0.08  (-0.27,0.13) 

-0.11  (-0.3,  0.1) 

T-score 

n=89  p=0.545 

n=89  p=0.217 

n=93  p=0.474 

n=93  p=0.303 

NAB  Digit:  Backward  raw 

-0.2  (-0.4,  0.01) 

0.49  (0.31,0.64) 

0.16  (-0.05,  0.36) 

-0.06  (-0.27,0.15) 

n=84  p=0.066 

n=84,  p<0.001* 

n=88p=0.142 

n=88  p=0.552 

NAB  Digit:  Backward  T- 

-0.27  (-0.46,  -0.06) 

0.46  (0.28,  0.61) 

0.07  (-0.14,  0.27) 

-0.08  (-0.28,0.13) 

score 

n=87,  p=0.011* 

n=87,  p<0.001* 

n=91  p=0.537 

n=91  p=0.463 

CTMT :  Composite  Index 

0  (-0.2,  0.21) 

0.26  (0.05,  0.44) 

0.01  (-0.19,  0.22) 

0.05  (-0.16,  0.25) 

n=88  p=0.963 

n=88,  p=0.015* 

n=92  p=0.916 

n=92  p=0.632 

WRAT :  Raw  score 

0.04  (-0.16,  0.24) 

0.39  (0.21,0.55) 

0.14  (-0.05,  0.33) 

-0.08  (-0.27,0.11) 

n=99  p=0.697 

n=99,  p<0.001* 

n=103  p=0.145 

n=103  p=0.416 

WRAT :  Standardized  score 

0.02  (-0.18,0.21) 

0.36  (0.18,0.52) 

0.15  (-0.04,  0.34) 

-0.08  (-0.27,0.11) 

n=99  p=0.869 

n=99,  p<0.001* 

n=103  p=0.121 

n=103  p=0.404 

SRT:  Baseline 

0.19  (-0.02,  0.38) 

-0.14  (-0.34,  0.07) 

-0.2  (-0.39,  0.01) 

0.08  (-0.12,  0.28) 

n=90  p=0.08 1 

n=90  p=0.181 

n=9 1  p=0.064 

n=91  p=0.434 

SRT:  End  of  testing 

0.21  (0,  0.41) 

-0.12  (-0.32,  0.1) 

-0.15  (-0.35,0.06) 

0.1  (-0.12,  0.3) 

n=84  p=0.053 

n=84  p=0.286 

n=86  p=0.161 

n=86  p=0.369 
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AIM  4:  DETERMINE  ABILITY  OF  DUAL-TASK  AND  MULTITASK  TEST  ITEMS  TO 
DISCRIMINATE  BETWEEN  HEALTHY  CONTROL  SM  AND  SM  WITH  MTBI  SYMPTOM 
COMPLEX. 

As  described  above,  the  AMMP  battery  of  6  tasks  was  administered  to  healthy  SM  (controls)  and  those 
with  known  mild  traumatic  brain  injury  symptom-complex.  Except  for  the  inertial  sensor  findings  on  the  ISAW- 
grid  task  which  was  unchanged  throughout  the  iterative  reliability  process,  data  were  collected  on  SM  stationed  at 
Fort  Bragg.  The  inertial  sensor  data  analysis  for  the  ISAW-grid  including  findings  from  HC  subjects  from 
USARIEM  as  well  as  Fort  Bragg.  Findings  on  between-group  differences  are  reported  first  for  AMMP  multitasks 
(CQ  Duty,  Run-Roll-Aim,  Patrol-Exertion)  and  then  AMMP  dual-tasks  (Illinois  Agility-Packing  List,  Load 
Magazine-Radio  Chatter,  ISAW-Grid,). 

Known  Groups  Methods 

Data  source 

Tasks  were  administered  to  participants  and  score  sheets  were  completed.  Analysis  for  each  task  includes 
subjects  who  completed  that  task.  For  certain  tasks,  data  collected  as  part  of  evaluating  IRR  were  included  for 
items  that  did  not  change  and  had  high  IRR.  This  included  CQ  Duty  (number  of  visits  and  total  time),  Load- 
Magazine-Radio  Chatter,  and  ISAW-grid  tasks.  Since  these  tasks  from  IRR  were  scored  by  more  than  one 
individual,  a  data  set  to  be  used  for  known  groups  analysis  was  selected  randomly  from  among  the  2  or  3  raters. 

Analysis 

Tests  for  significant  differences  between  groups  for  continuous  data  included  the  t-Test  for  independent 
groups  to  test  for  difference  of  means  if  the  data  met  the  criteria  for  a  normal  distribution,  otherwise,  the  Mann- 
Whitney  U  test  for  independent  groups  was  used  to  test  the  difference  in  medians.  In  most  cases,  data 
distributions  did  not  meet  criteria  for  normal  distribution  as  tested  with  the  Shapiro-Wilk  Test.  Chi-Square  was 
used  to  test  for  significant  differences  between  groups  for  dichotomous  variables  such  as  number  completing  the 
task.  A  two  sided  p-value  <  0.05  was  required  for  significance.  SPSS  V22.0  (IBM  Inc.)  was  used  for  analysis. 

Effect  sizes  were  calculated  for  task  items  that  were  significantly  different  between  groups.  Non- 
parametric  effect  sizes  were  calculated  as;  z/sqrt(n),  z=  calculated  z-score  based  on  ranks,  n=  total  of  HC  and 
mTBI9.  In  reading  this  document,  be  aware  that  cut  points  for  non-parametric  effect  sizes  are:  0.1  as  small,  0.3  as 
medium,  and  0.5  as  large.  Cohen’s  d  effect  size  based  on  mean  and  standard  deviation,  has  cut  points  that  are 
greater  at:  small  as  0.2,  medium  as  0.5,  large  as  0.8. 


CHARGE  OF  QUARTERS  (CQ)  DUTY 

Introduction 

Performance-based  assessments  that  place  demands  on  multitasking  are  known  to  be  sensitive  to 
executive  dysfunction.  The  CQ  Duty  task  was  developed  to  target  executive  dysfunction  using  a  military  type  task 
scenario. 

Description 

CQ  Duty  is  a  performance-based  multitasking  assessment  that  was  developed  using  the  structure  and 
metrics  of  the  Multiple  Errands  Test  (MET)10.To  perform  the  CQ  Duty  task,  SM  are  given  a  list  of  tasks  to  carry¬ 
out  as  efficiently  as  possible  using  supplies  and  information  from  4  taped-off  work  areas.  The  primary  tasks 
include  reporting  a  duty  shift  change,  assembling  a  footstool  from  PVC  pipe,  reporting  information  to  a  superior 
at  various  times,  and  inventorying  supplies  while  remembering  to  carry-out  embedded  prospective  memory 
assignments3.  Performance  is  characterized  in  terms  of  the  following  metrics:  accurate  task  completion; 
performance  time;  number  of  visits  between  the  4  work  areas  made  to  complete  the  exercise;  and  the  number  of 
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times  the  subject  broke  task  rules.  See  details  of  task  set  up,  test  administration  and  scoring  in  the  AMMP  Manual 
(Appendix  4). 

Rationale 

Performance-based  assessments  that  place  demands  on  multitasking  are  known  to  be  sensitive  to 
executive  dysfunction11.  The  MET  is  a  widely  used  research  metric  that  was  designed  to  measure  executive 
dysfunction  in  community  dwelling  individuals  with  mild  cognitive  impairment10.  Patients  are  required  to 
structure  their  completion  of  a  series  of  interleaving  tasks  in  the  hospital  lobby12"14  or  a  shopping  center13,  without 
engaging  the  examiner.  The  MET  has  been  shown  to  be  sensitive  to  executive  dysfunction  after  both  traumatic 
brain  injury12  and  mild  stroke14  but  has  limited  clinical  feasibility15. 

Similar  to  the  MET,  the  CQ  Duty  places  demands  on  multitasking  based  on  the  definition  of  that  term 
proposed  by  Burgess16,  who  proposed  that  multitasking  has  the  following  key  elements  including  many  discrete 
but  interleaving  tasks  that  have  to  be  completed  one  at  a  time;  interruptions  and  unexpected  outcomes  occur;  and 
involve  delayed  intentions  that  place  demands  on  prospective  memory.  However,  the  CQ  Duty  was  designed  to 
overcome  the  MET’s  difficulties  with  clinical  feasibility  by  keeping  test  administration  to  around  30  minutes  and 
allowing  test-task  set  up  in  most  clinical  environments. 

Hypotheses/expectations 

We  hypothesized  that  the  CQ  Duty  primarily  places  demands  on  executive  functioning,  which  would 
most  likely  be  evidenced  in  the  number  of  visits  to  the  4  work  areas  taken  during  the  exercise  (less  visits 
indicative  of  greater  foresight  and  planning).  Similarly,  we  hypothesized  some  correlation  between  the 
aforementioned  metric  and  number  of  moves  on  the  TOH17,  CTMT18,  both  measures  of  executive  functioning,  and 
the  WRAT  4  Reading  Test19,  a  proxy  for  intelligence  and  education  level. 

Method 

As  part  of  the  AMMP  validation  study  to  evaluate  IRR  and  known-group  discrimination,  the  CQ  Duty 
was  administered  to  SM  with  mTBI  and  HCs.  CQ  Duty  assessment  solely  relied  on  observational  metrics,  as 
described  above. 

Between-groups  analyses  for  number  of  visits  and  performance  time  were  conducted  on  51  HCs  and  51 
SM  with  mTBI.  Between-groups  analyses  for  task  performance  and  rule  breaks  were  conducted  on  50  HCs  and  33 
SM  with  mTBI.  As  described  earlier,  an  iterative  approach  to  scoring  refinement  was  used  to  optimize  inter-rater 
reliability  in  the  first  half  of  the  study.  Performance  time  and  number  of  visits  remained  consistent  throughout  the 
entire  data  collection  period,  thus  there  were  larger  numbers  in  these  analyses,  whereas  there  small  changes  were 
made  in  task  performance  and  rule  break  scoring  methods.  Finally,  between  group  analyses  are  provided  with  and 
without  data  from  Subject  83.  This  HC  took  51  visits  to  complete  the  task,  an  outlier  by  either  group  definition. 

Results 


As  summarized  in  Table  IVA,  study  findings  suggest  statistically  significant  differences  between  known 
groups  of  HC  SM  and  SM  with  mTBI  in  task  performance  accuracy  and  number  visits  taken  to  the  4  work  areas 
to  complete  the  exercise  (going  forward,  referred  to  a  “number  of  visits”).  However,  effect  sizes  (calculated  using 
Cohen’s  d)  were  relatively  small.  There  were  no  group  differences  between  performance  time  and  rules  broken. 

As  hypothesized,  there  appear  to  be  a  small,  statistically  significant  correlations  between  CQ  Duty  and 
reading  ability  (WRAT  4)  and  neurocognitive  measures  of  executive  functioning  (TOH  and  CTMT)  (Table  IVB). 
Although  these  correlations  are  significantly  different  from  zero,  their  magnitude  indicates  limited  influence  on 
performance  metrics. 

In  order  to  evaluate  order-effects,  28  of  the  51  HC  subjects  and  29  of  the  51  mTBI  subjects  performed  the 
CQ  Duty  is  the  first  half  of  their  testing  session  (either  first,  second  or  third)  while  the  remainder  of  the  subjects 
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performed  the  CQ  Duty  in  the  second  half  of  their  session  (either  fourth,  fifth,  or  sixth).  There  was  no  evidence  of 
an  order  effect  (p  =  0.842). 

Finally,  there  is  additional  evidence  that  the  CQ  Duty  can  identify  outliers  relative  to  the  performance  of 
HCs.  A  distribution  free  method  for  identification  of  outliers  was  used  due  to  the  non-normality  of  the  data.  Using 
the  interquartile  range  (IQR)  for  the  HCs  defined  as  the  25th  to  75th  percentiles,  the  cut  point  for  the  outlier  was 
calculated  using  Tukey’s  definition  of  1.5  times  the  range  of  the  IQR  (i.e.  if  the  25th  percentile  started  at  10  and 
the  75th  percentile  ended  at  16,  the  range  is  6  (16-10)  and  the  outlier  cut  point  would  be  25  (16  +  6*1.5).  Five 
subjects  with  mTBI  were  identified  as  outliers  based  on  Total  number  of  visits  using  the  Tukey  criteria  compared 
to  zero  HCs.  (See  Section  V  for  more  information  on  identification  of  outliers). 


Table  IVA.  Between-group  comparisons  of  key  CQ  Duty  metrics 


Metric 

(#HC,  #mTBI) 

HC 

Mean  (SD) 
Median  (Range) 

mTBI 

Mean(SD) 

Median(Range 

) 

p-value,  method 

Task  performance  (50,33) 

Remove  case  with  51  visits 
(ID83- Control)  (49,33) 

34.4  (2.8) 

35  (29-38) 

34.5  (2.8) 

35  (29-38) 

32.6(3.9) 

34(21-37) 

0.020,  Mann- Whitney 
Effect  size: 0.25 

0.017,  Mann- Whitney 
Effect  size: 0.25 

#  of  Visits(51,51) 

Remove  case  with  51 
(ID83-HC)  visits  (50,51) 

12.6  (6.7) 

11  (6-51) 

11.86  (3.8) 

11  (6-22) 

14.4(5.3) 

13(7-30) 

0.027,  Mann- Whitney 
Effect  size: 0.22 

0.016,  Mann-Whitney 
Effect  size: 0.24 

Time  (mins)  (51,51) 

Remove  case  ID83  (50,51 

18.9  (4.8) 

18  (11.7-31.9) 

18.6  (4.5) 
18.0(11.7-30.8) 

20.1(4.9) 

19.3(13.1-37.0) 

0.171,  t-test 

0.1 2  fit-test 

n  (%  of  total) 

n  (%  of  total) 

Total  rules  broken  (50,33) 

Remove  case  ID83  (49,33) 

0  rules:33,(66.0%) 

1  rule:  14  (28.0%) 

2  rules: 3  (6.0%) 

0:20  (60.6%) 
1:9  (27.3%) 

2:4  (12.1%) 

0.687,  chi-square, 
Monte-Carlo  Simulation 
0.683 

Total  Times  Rules 

Broken>3  (50,33) 

Remove  case  ID83  (49,33) 

7  (14.0%) 

7  (14.3%) 

5  (15.2%) 

1,  Fisher  Exact  Test 

1 ,  Fisher  Exact 

Complete  (52,51) 

51  (98.1%) 

100% 

1 ,  Fisher  exact 

Table  IVB.  Correlation  between  CQ  Duty  metrics  and  neurocognitive  measures  (Pearson  correlation  coefficients) 


Neurocognitive 

measures 

Task  performance 
accuracy  total  score 

Total  #  of  visits  to 
complete  the  task 

Total  time  to  perform 
the  task 

Tower  of  Hanoi: 
number  of  moves 

-0.3,  p=0.006 
n=83 

0.36,  p<0.001 
n=92 

0.21,  p=0.041 
n=92 

CTMT:  Sum 

0.28,  p=0.015 
n=76 

-0.21,  p=0.045 
n=91 

-0.4,  p<0.001 
n=91 

CTMT:  Composite 
index 

0.28,  p=0.015 
n=76 

-0.21,  p=0.041 
n=91 

-0.4,  p<0.001 
n=91 

WRAT :  Raw  score 

0.29,  p=0.007 
n=83 

-0.25,  p=0.01 
n=102 

-0.22,  p=0.024 
n=102 

Discussion 
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These  data  suggest  that  the  number  of  visits  needed  to  perform  the  CQ  Duty  and  total  accuracy  of  task 
performance  appear  to  distinguish  between  groups  of  HC  SM  and  those  who  with  known  mTBI  who  are  receiving 
rehabilitation  services  for  mTBI-related  symptoms.  However,  the  effect  sizes  were  relatively  small.  As 
hypothesized,  there  was  a  small  correlation  between  these  metrics  and  neurocognitive  measures  specific  to 
executive  functioning  and  intelligence/education  level.  All  told,  these  data  suggest  that  the  CQ  Duty  may  help 
identify  existing  problems  with  executive  functioning  that  may  be  a  barrier  to  return  to  duty.  These  findings  are 
similar  to  that  involving  with  MET  and  individuals  with  traumatic  brain  injury12  and  mild  stroke14. 

The  2  groups  of  SM  understudy  had  a  wide  range  of  within-group  characteristics,  which  may  have 
contributed  to  the  small  effect  sizes.  For  example,  the  research  team  did  not  have  specific  information  regarding 
the  symptomatology  for  which  the  SM  with  mTBI  were  receiving  rehabilitation  services.  Therefore,  within  the 
mTBI  group,  SM  may  not  have  had  lingering  executive  dysfunction  but  have  had  sensorimotor  difficulties 
unlikely  to  be  challenged  by  the  CQ  Duty.  Further  study  of  the  CQ  Duty  is  needed  to  evaluate  its  sensitivity  and 
specificity.  Such  information  could  be  valuable  in  determining  whether  or  not  the  CQ  Duty  is  a  useful  metric  for 
identifying  SM  with  mTBI  who  would  benefit  from  cognitive  rehabilitation. 

Further  study  is  also  needed  to  fully  understand  the  relationship  between  intelligence/education  level,  CQ 
Duty  performance,  and  relevance  for  RTD  decision-making.  Beyond  executive  functioning,  the  CQ  Duty  places 
obvious  demands  on  reading  comprehension.  While  the  correlation  is  relatively  small,  there  appears  to  be  an 
association  between  CQ  Duty  performance  and  intelligence/education  level,  which  may  or  may  not  be  relevant  to 
return  to  duty  decision-making  for  all  military  occupational  specialties. 

Recommendations 

Further  study  is  needed  to  evaluate  the  responsiveness  to  change  or  improvement  with  intervention,  and  to 
sensitivity  and  specificity  of  the  CQ  Duty.  This  would  be  particularly  important  in  confirming  its  ability  to 
identify  existing  executive  dysfunction  in  SM  with  presumably  resolved  mTBI  in  the  context  of  return  to  duty 
decision-making.  Studying  responsiveness  to  change  would  require  an  alternate  form  of  this  AMMP  task  as  once 
this  task  is  completed,  the  subject  would  likely  retain  the  solution  invalidating  repeated  testing  with  the  same  task. 


PATROF-EXERTION  MULTITASK 

Introduction 

The  Patrol-Exertion  task  is  a  performance-based  multitasking  assessment  that  was  developed  using  the 
broad  dictionary  definition  of  multitasking  while  incorporating  multiple  known  vulnerabilities  following 
concussion/mTBI.  During  the  Patrol-Exertion  task,  the  subject  is  challenged  to  gather  information  from  video 
surveillance  and  radio  communications  while  exercising  at  65  to  85%  of  the  subject’s  age  predicted  maximal  heart 
rate  by  doing  continuous  step-ups  on  an  exercise  step  to  simulate  a  dismounted  patrol.  IED  markers  and  pertinent 
logistical  information  must  be  recalled  and  reported  at  specific  times  while  also  requiring  a  reaction  time  trigger 
switch  press  to  an  intermittently  occurring  tone  sound. 

Description 

The  SM  will  be  fitted  with  a  Polar  Heart  Rate  (HR)  monitor  (chest  strap)  with  the  evaluator  holding  the 
wrist  watch  sized  monitor  to  follow  the  heart  rate,  an  Army  combat  helmet  and  eye  protection  (clear  lenses),  and 
hold  a  simulated  weapon  to  increase  realism  of  the  condition.  The  simulated  weapon  will  have  a  small  switch 
attached  to  the  hand  grip  and  a  small  speaker  attached  to  upper  barrel  of  the  simulated  weapon.  Two  30  second 
baseline  trials  of  randomly  occurring  tones  requiring  a  reaction  time  trigger  press  occur  before  starting  the 
patrolling  video.  The  first  trial  occurs  while  the  subject  is  standing  still  and  the  second  trial  occurs  while  the 
subject  is  stepping  on  the  aerobics  step.  A  total  of  1 1  tones  will  sound  at  preset  times  during  the  video 
instructions  on  IED  markers  and  the  virtual  patrolling  scenario  requiring  the  SM  to  press  the  switch.  This  is  done 
to  assess  the  SM  reaction  time  during  a  cognitive  and  exertional  activity.  The  participant  will  listen  to  about  2-3 
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minutes  of  instructions  and  then  begin  continuous  stepping  onto  a  4”  step  platform  to  increase  heart  rate  to 
between  65-85%  of  his  or  her  age  predicted  HR  maximum  to  require  moderate  exertion  that  could  increase 
exertional  symptoms  associated  with  mTBI.  During  the  continued  stepping,  the  participant  watches  an 
approximately  12  minute  virtual  reality  scene  created  for  the  project  by  the  developers  of  Virtual  Afghanistan 
(Institute  for  Creative  Technologies  University  of  Southern  California).  Approximately  the  first  5  minutes  of  the 
video  includes  instructions  for  the  tactical  pause  reports  on  IED  markers  that  they  might  see  in  the  video  and  the 
Patrol  Report  to  be  given  at  the  end  of  the  video.  The  SM  is  told  to  watch  for  activities  on  the  video  that  are 
relevant  to  a  post-patrol  report.  Cues  are  provided  if  necessary  to  maintain  heart  rate  in  the  target  range  of  65-85% 
of  age-predicted  maximal  heart  rate.  If  unable  to  exert  at  the  desired  range,  the  participant  is  permitted  to 
complete  the  assessment  at  his  or  her  own  pace  or  if  necessary,  to  stop  the  test.  Following  the  video  scenario,  the 
participant  will  answer  questions  in  support  of  a  post-patrol  report  into  a  recording  microphone.  The  report  is 
analyzed  for  correctness  of  responses  for  both  the  tactical  pause  IED  marker  reports  and  the  reconnaissance  or 
post-patrol  report  questions.  Reaction  times  are  recorded  by  Bluetooth  onto  a  laptop  using  in  house  developed 
JAVA  software  (see  training  manual  materials  for  this  task.  Visual  attention,  memory  and  decision-making 
under  moderate  exertional  conditions  are  often  impaired  following  mTBI.  This  task  places  demands  on  divided 
and  alternating  attention,  working  memory,  visual  attention,  vertical  gaze  stability,  visual  scanning,  reaction  time, 
auditory  and  visual  processing  under  a  condition  of  moderate  exertion.  This  test  takes  23-25  minutes  to  perform 
including  instructions  and  set  up  of  subject. 

Rationale 

This  multitask  is  based  on  the  general  definition  of  multitasking  from  the  dictionary  (American  Heritage® 
Dictionary  of  the  English  Language,  Fifth  Edition.  Copyright  ©  201 1  by  Houghton  Mifflin  Harcourt  Publishing 
Company.  Published  by  Houghton  Mifflin  Harcourt  Publishing  Company)  which  states  that  multitasking  is 
engaging  in  more  than  one  activity  at  the  same  time  or  serially  switching  one’s  attention  back  and  forth  from  one 
activity  to  another.  In  developing  the  Patrol-Exertion  task,  the  AMMP  team  focused  on  including  a  number  of  the 
known  vulnerabilities  following  mTBI  including  exertional  intolerance  and  impaired  reaction  time  as  well  as 
impaired  gaze  stability  in  the  setting  of  an  operationally  relevant  patrol  task  requiring  cognitive  engagement  and 
decision-making.  This  task  places  demands  on  divided  and  alternating  attention,  working  memory,  visual 
attention,  vertical  gaze  stability,  visual  scanning,  reaction  time,  auditory  and  visual  processing  under  a  condition 
of  moderate  exertion. 

Hypotheses/expectations 

The  Patrol-Exertion  task  is  a  multitask  requiring  multiple  concurrent  tasks  including  exertion  (to  65-85% 
of  age  predicted  maximum  heart  rate)  by  continuous  stepping  on  an  aerobics  step,  auditory  and  visual  processing 
and  reporting  relevant  information  and  reacting  to  a  tone  as  fast  as  possible.  The  expectation  was  for  a  decrement 
in  the  ability  to  identify  IED  markers  and  answer  post  patrol  questions  as  well  as  slower  reaction  times  in  subjects 
with  mTBI  as  compared  to  HCs.  Given  that  this  task  requires  sustained  attention  with  some  executive  function,  it 
was  expected  to  correlate  to  standard  neurocognitive  tests  of  attention  and  memory  including  the  CTMT,  the  NAB 
Digits  forward/Digits  backward,  NAB  Numbers  and  Letters,  and  the  TOH.  A  correlation  with  WRAT  Reading 
was  not  necessarily  hypothesized;  however,  the  expectation  would  be  that  a  subject  who  is  of  higher  intelligence 
may  be  more  able  to  develop  a  strategy  for  being  successful  at  this  task.  As  this  task  evaluated  reaction  time  in  a 
complex  context,  the  reaction  time  metric  was  expected  to  correlate  with  the  standard  SRT  test,  part  of  the 
ANAM. 

Method 

As  part  of  the  AMMP  validation  study  to  evaluate  IRR,  known  group  discrimination,  and  convergent  and 
discriminate  validity,  the  Patrol-Exertion  task  was  administered  to  SM  with  mTBI  and  HCs.  The  Patrol-Exertion 
task  assessment  relied  on  both  instrumented,  subject  reported  and  observational  metrics,  including  number  of  IED 
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markers  identified  and  post  patrol  questions  answered.  The  reaction  time  component  of  the  task  was  instrumented 
and  the  number  of  milliseconds  to  react  was  transferred  via  Bluetooth  technology  and  recorded  from  a  computer 
screen.  Subjects  were  asked  to  verbally  rate  their  visual  clarity  on  a  0  (normal,  clear  and  stable  vision)  to  10 
(extremely  blurry  or  jumpy  vision  “the  worst  it  could  be”).  Subjects  were  asked  to  rate  how  hard  they  were 
working  using  the  standard  rate  of  perceived  exertion  scale  (6-20  Scale, 

http://www.cdc.gov/phvsicalactivitv/basics/measuring/exertion.htm).  Between-groups  analyses  for  these  metrics 
were  conducted  on  51  HCs  and  35  SM  with  mTBI.  As  this  AMMP  task  required  multiple  revisions,  not  all 
subjects  tested  early  in  the  Fort  Bragg  testing  were  evaluated  on  all  components  of  Patrol-Exertion  metrics.  See 
Table  IVC  for  the  specific  number  of  subjects  tested  on  each  metric. 

Results 

Non  parametric  between  groups  analysis  revealed  significant  differences  in  the  report  of  issues  with 
visual  clarity  and  in  the  reaction  time  both  at  the  beginning  and  the  end  of  the  task  between  HCs  and  concussed 
personnel  (Table  IVC).  The  correct  responses  to  IED  markers  and  post  patrol  questions  and  the  report  of  “how 
hard  are  you  working”  (using  Rate  of  Perceived  Exertion  Scale)  did  not  distinguish  groups.  As  discussed  under 
Aim  2,  the  significant  correlations  between  the  total  correct  responses  to  IED  marker  reporting  and  post  patrol 
questions  to  the  error  percentile  of  the  TOH  and  to  the  WRAT  Reading  score  did  not  meet  the  benchmark  of  > 
failing  to  support  the  concept  that  reported  responses  had  some  correlation  to  executive  function  and  to  a  proxy 
for  measure  of  intelligence  and  education.  A  possible  reason  for  this  may  be  due  to  the  patrolling  video  used  for 
this  task  which  was  not  highly  complex  or  difficult  to  follow.  The  correct  responses  to  IED  markers  and  post 
patrol  questions  also  did  not  correlate  to  the  measures  of  attention  and  memory  (CTMT  and  NAB).  SRT 
measured  at  the  end  of  the  AMMP  battery  was  used  as  a  measure  of  fatigue.  This  reaction  time  did  correlate  to 
the  reaction  times  tested  in  this  task  both  at  the  beginning  of  stepping  (first  half  of  video)  and  at  the  end  of 
stepping  (second  half  of  video)  (Table  IVC  and  Aim  2,  Table  IIID(3)). 

Discussion 

Subjects  with  mTBI  had  slower  reaction  times  and  more  reports  of  poor  visual  clarity  with  jumpy  and 
blurry  vision  than  HC.  Reaction  times  at  baseline  either  standing  or  stepping  without  watching  the  video  were  not 
significantly  different  between  the  groups,  while  those  tested  during  the  video  (watching  video,  stepping  to 
simulate  exertion,  and  responding  to  reaction  time  tone  sounds)  did  distinguish  the  groups;  supporting  the  concept 
that  multiple  task  overlays  tax  a  limited  and  shareable  amount  of  brain  resources  that  may  be  limited  as  a  result  of 
the  mTBI.  As  these  are  performance  based  assessments,  the  mechanism  or  cause  of  the  findings  cannot  be 
determined.  It  is  speculated  that  given  the  number  of  visual  system  and  vestibular  complaints  reported  by 
subjects  with  mTBI  (see  patient  demographic  information  Tables  B3  and  B5)  that  vestibular  system  residual 
deficits  following  mTBI  may  play  a  role  in  the  these  findings. 

Recommendations 

The  next  step  would  be  to  evaluate  the  Patrol-Exertion  task  on  subjects  who  are  at  the  point  of  being 
ready  to  return  to  duty  to  see  if  it  correlates  well  with  standard  RTD  assessments  and  predicts  successful  or  failed 
duty  abilities.  It  is  recommend  that  consideration  be  made  to  modify  the  patrolling  video  used  in  this  task  to  a 
busier,  louder  and  more  complex  environment  requiring  more  focused  attention  with  additional  executive  function 
or  decision-making  requirements.  These  changes  may  also  improve  the  correlation  of  task  metrics  to  standard 
neurocognitive  tests  for  executive  function,  IQ  and  attention.  If  this  task  is  to  be  used  to  inform  readiness  to 
return-to-duty,  a  longitudinal  study  exploring  the  predictive  validity,  and  information  on  normative  data  on  HCs 
(ultimately  would  want  to  establish  cut-points  for  definite  “Go”  and  “No  Go”  parameters)  are  needed.  If  it  is  to 
be  used  to  measure  progress  for  RTD,  responsiveness  to  change  as  a  SM  recovers  spontaneously  or  responds  to 
treatment  should  be  evaluated,  this  would  require  an  alternate  form  patrolling  video.  Further  study  is  needed  to 
evaluate  the  sensitivity  and  specificity  of  the  Patrol-Exertion  task. 
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Table  IVC.  Patrol-Exertion  Multitask  Between  Group  Findings  and  Correlations 


Patrol-  Exertion 

Scoring 

Reliability 

Known  Groups 
Analysis 

Correlations  to  Standard  Neurocognitive  tests 

Metrics 

Fort  Bragg 

HC 

mTBI 

ICC  (95%  Cl),  n,  p-value 

n  =  26 

19  mTBI,  7  HC 

Mean  (SD), 

Median  (Range) 

Mean  (SD), 

Median  (Range) 

p-value 

ICC  (95%  Cl) 

TOH  #  moves 

WRAT  Reading  (IQ) 

IED  &  Patrol 
Questions  correct 

0.97  (0.95-0.99) 

N=51 

18.2(2.6),  19(11-22) 

N=33 

17.5  (2.8)  ,18  (10-22) 

0.179 

-0.12  (-0.33, 

0.09) 

n=84  p=0.262 

0.26  (0.05,  0.45) 
n=84,  p=0.016* 

Vision  Clarity 
initial 

0.99  (0.97-1) 

N=45 

0.13  (0.4),  0(0-2) 

N=25 

0.79  (2.3),  0  (0-5) 

0.002, 

0.06  (-0.17,0.3) 
n=70  p=0.595 

0.05  (-0.19,  0.28) 
n=70  p=0.698 

Vision  Clarity  end 

0.99  (0.98-1) 

N=51 

0.4  (0.9),  0  (0-4) 

N=33 

1.6  (2.0),  1  (0-8) 

<0.001 

0.15  (-0.07,  0.35) 
n=84  p=0.17 

0.1  (-0.12,  0.31) 
n=84  p=0.364 

RPE  end 

1.0  (1-1) 

N=51 

11.2  (2.0),  11  (7-15) 

N=33 

12.1  (2.3),  12(7-17) 

0.133 

0.07  (-0.15,0.28) 
n=84  p=0.546 

-0.1  (-0.31,0.12) 
n=84  p=0.373 

Reaction  time  - 
early 

(mean  of  6) 

* 

N=48 

590.4  (224.1), 

538.8  (297.5-1305) 

N=32 

681.6  (216.9), 

640  (370-1400) 

0.013 

-0.02  (-0.24,  0.2) 
n=80  p=0.85 

-0.21  (-0.41,0.01) 
n=80  p=0.056 

Reaction  time-late 
(mean  of  5) 

* 

N=48 

547.1  (209.7), 
535.2(334-930) 

N=33 

674.1  (209.7), 

602  (471-1400) 

0.002 

0.02  (-0.2,  0.23) 
n=81  p=0.882 

-0.1  (-0.31,0.12) 
n=8 1  p=0.358 
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RUN-ROLL-AIM 

Introduction 

This  Run-Roll-Aim  task  is  a  performance-based  multitasking  assessment  that  was  developed  using  the 
broad  dictionary  definition  of  multitasking  (engaging  in  more  than  one  activity  at  the  same  time  or  serially 
switching  one’s  attention  back  and  forth  from  one  activity  to  another)  while  incorporating  multiple  known 
vulnerabilities  following  concussion/mTBI,  specifically  tasks  that  challenge  the  vestibular  system. 

Description 

The  SM  completes  a  high  level  mobility  task  with  multiple  maneuvers  while  carrying  a  simulated 
weapon.  Maneuvers  are  cued  by  a  remotely  controlled  computer  screen,  requiring  a  rapid  start,  running,  avoiding 
a  “trip  wire”  obstacle,  3-5  second  rush,  combat  rolling,  search  for  visual  targets  through  a  short  focal  point  (insect 
or  bird  viewing)  scope  mounted  in  place  of  a  weapon  scope  on  a  mock  M-16,  rapid  lateral  dodging  and  back 
pedaling.  This  task  requires  the  SM  to  demonstrate  high  level  balance  and  mobility  skills  not  unlike  those 
required  in  a  battlefield  situation,  alternating  between  quick  position  changes  and  focused  visual  search  through  a 
weapon  scope.  Rapid  head  and  body  position  changes  stimulate  the  vestibular  system,  so  SM  with  vestibular 
impairment  may  have  difficulty  with  this  task.  This  test  takes  approximately  13-14  minutes  to  perform. 

Rationale 

During  initial  task  development  and  end-user  feedback  we  devised  a  new  task  (Run-Roll-Aim)  that 
involves  maneuvers  known  to  be  used  in  military  training  that  might  provoke  the  vestibular  system  (rolling, 
speeded  movement  from  one  position  to  another).  We  also  expected  that  dynamic  visual  acuity  could  be  impaired 
during  movement,  so  we  included  a  “trip  wire”  obstacle.  Given  the  potential  for  vestibular  system  impairment,  we 
incorporated  a  visual  search  task  through  a  scope  on  a  simulated  weapon. 

Researchers  that  study  older  adult  cognition  during  mobility  tasks  have  used  a  “directional  Stoop” 
component  to  walking  tasks  that  presents  two  stimuli  to  direct  movement.  An  arrow  (an  automatic  cue)  and  a 
letter,  R  or  L  (less  automatic  cues)  are  presented  together.  Subjects  are  instructed  to  follow  the  letter  in  the 
direction  of  the  roll.  Both  congruent  (R  and  right  pointing  arrow)  and  incongruent  (R  with  left  pointing  arrow)  are 
presented,  necessitating  4  trials  to  provide  congruent  and  incongruent  cues  in  both  directions. 

This  task  does  not  meet  all  of  the  conditions  that  are  described  for  multitask  scenarios,  as  the  sequence  of 
maneuvers  is  cued  by  a  computer  screen  and  there  is  one  way  to  accomplish  the  task,  however,  it  does  incorporate 
several  cognitive  functions  (remember  the  target  -  odd  or  even  numbers,  inhibit  automatic  response  to  directional 
Stroop).  During  pilot  testing,  this  task  is  one  that  SM  clearly  enjoyed  completing  given  the  face  validity  of  the 
combined  skills  is  strong.  It  challenges  mobility  in  a  way  that  is  unlike  any  other  balance  test  that  is  in  use  in 
clinical  practice. 

Since  dual-task  movement  impairments  post-concussion  have  only  been  detected  using  laboratory 
measures,  we  incorporated  the  use  of  wireless  inertial  sensors  on  the  head  and  torso  so  that  we  might  compare 
more  instrumented  measures  between  groups  in  the  event  that  the  simple  observational  measures  did  not  detect 
differences. 

Hypotheses/expectations 

We  expected  that  multiple  components  of  the  Run  Roll  Aim  task  could  present  challenges  for  individuals 
with  mTBI.  Given  the  Stroop  effect  requires  inhibition  of  automatic  responses  in  the  incongruent  condition,  we 
expected  that  individuals  with  mTBI  who  may  have  reduced  frontal  inhibitory  function,  may  have  more  difficulty 
with  the  incongruent  trial  cues  (longer  delay  in  response,  rolling  the  wrong  direction,  etc).  Individuals  with 
vestibular  impairment  as  a  result  of  mTBI  we  be  expected  difficulty  with  dynamic  transitions,  with  dynamic 
visual  acuity  and  with  visual  search  and  thus  score  lower  than  HC  on  task  metrics. 
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Method 


As  part  of  the  AMMP  validation  study  to  evaluate  inter-rater  reliability  and  known-group  discrimination, 
the  Run-Roll-Aim  was  administered  to  SM  with  mTBI  and  HCs.  Observational  and  inertial  sensor  data  were 
collected.  Between-groups  analyses  for  trial  and  total  time  (all  4  trials)  and  odd/even  numbers  and  task 
components  (miss  trip  wire,  roll  direction,  etc.)  correct  and  errors  in  each  trial  and  in  total  (all  4  trials)  were 
conducted  on  50  HC  and  30  SM  with  mTBI.  Subjects  with  mTBI  tested  initially  at  Fort  Bragg  were  not  given  a 
practice  trial  so  there  data  was  not  included  in  the  between  groups  analysis. 

Results 

Observational  data 

Observational  data  were  reliably  rated  with  the  exception  of  the  total  error  ratings.  This  is  in  part  because 
errors  were  few  in  number  on  the  task  in  general,  so  with  a  reduced  range,  ICC  values  are  affected.  In  addition, 
the  attempt  to  characterize  the  Stroop  response  during  the  fast  paced  task  was  problematic.  We  expect  that  an 
additional  problem  with  reliability  relates  to  the  rating  of  a  “hesitation”  of  more  than  1  second  associated  with 
Stroop  response.  This  is  difficult  to  judge  reliability  just  by  watching  unless  the  delay  is  longer.  We  plan 
additional  analyses  of  this  element  of  the  scoring  prior  to  publication  of  findings  about  this  task. 

The  observational  data  from  Run-Roll-Aim  were  not  sensitive  to  group  differences  (see  Table  IVD).  The 
relatively  gross  nature  of  performance  that  one  can  observe  and  rate  by  observations  is  likely  a  reason  for  this. 
There  were  group  differences  in  the  ability  to  complete  the  task,  as  a  4  individuals  with  mTBI  were  unable  to 
complete  practice  and  2  others  required  extended  rest  periods  between  tasks  and  all  4  test  trials  as  a  result  of 
increased  symptoms.  This  did  not  occur  with  any  of  the  HC  subjects. 
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Table  IVD.  Run-Roll-Aim  Multitask  Findings 


Scoring 

Reliability 

Known  Groups  Analysis* 

Correlations  to  Standard  Neurocognitive  tests 

Metrics 

Fort  Bragg 

HC 

mTBI 

ICC  (95%  Cl),  n,  p-value 

n  =  26 

19  mTBI, 

7  HC 

n  =  50 

Mean  (SD) 

Median  (Range) 

n  =  30 

Mean  (SD) 

Median  (Range) 

p-value 

ICC  (95%  Cl) 

Tower  of  Hanoi 
#  moves 

WRAT  Reading  (IQ) 

Trial  1  time 
(seconds) 

Not  evaluated 

0.80  (0.24) 

0.74  (0.54-1.8) 

0.80  (0.21) 

0.74  (0.57-1.4) 

0.893 

0.13  (-0.09,  0.34) 
n=80  p=0.241 

-0.15  (-0.36,  0.07) 
n=80  p=0.191 

Trial  1  correct 

0.999 

(0.997-1) 

13.5  (0.70) 
14(11-14) 

13.2(1.0) 

13  (9-14) 

0.159 

-0.11  (-0.32,  0.11) 
n=80  p=0.342 

0.09  (-0.13,0.31) 
n=80  p=0.417 

Trial  1  errors 

0.96 

(0.91-1) 

1.9  (1.3) 

2  (0-6) 

2.4  (2.3) 
2(0-11) 

0.317 

* 

* 

Total  time 
(4  trials) 

* 

2.9  (0.59) 

2.8  (1.9-4. 2) 

3.0  (0.57) 

3  (2.2-4. 3) 

0.515 

t-test 

0.11  (-0.11,0.32) 
n=80  p=0.317 

-0.19  (-0.4,  0.03) 
n=80  p=0.087 

Total  correct 
(4  trials) 

* 

4.9  (4.6) 

4  (0-25) 

5.4  (4.4) 

4  (0-16) 

0.438 

-0.11  (-0.32,  0.11) 
n=80  p=0.326 

-0.07  (-0.28,0.15) 
n=80  p=0.548 

Total  errors 
(4  trials) 

0.64 

(0.13-0.92) 

4.9  (4.6) 

4  (0-25) 

5.4  (4.4) 

4  (0-16) 

0.438 

0.11  (-0.12,  0.32) 
n=80  p=0.35 1 

0.05  (-0.17,0.27) 
n=80  p=0.663 

*Known  groups  analysis  with  Mann-Whitney  U  unless  otherwise  noted 
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Inertial  sensor  data 

As  a  component  of  an  exploratory  assessment  for  identifying  subtle  mobility  differences  not  detectible  with 
observational  metrics,  inertial  sensors  were  incorporated  into  this  task.  For  this,  the  SM  wears  an  inertial  sensor 
(triaxial  accelerometer,  gyroscope  and  magnetometer;  NexGen  Ergonomics)  on  an  adjustable  headband  positioned 
at  the  SMs  forehead  and  an  inertial  sensor  at  the  SM’s  lumbar  area.  Data  transmitted  wirelessly  from  this  sensor 
includes  specific  time  for  completion  of  components  of  maneuver,  maximal  angular  velocity  of  head  movement 
and  trunk  during  combat  roll  and  acceleration  values  for  all  components  of  movement. 

Inertial  sensor  data  were  sensitive  to  group  differences  (refer  to  those  analyses),  consistent  with  the 
laboratory  type  nature  of  measurement  necessary  to  detect  subtle  changes  in  movement  associated  with 
mTB I/concussion.  37  HC  and  37  mTBI  subjects,  who  did  all  4  runs  of  the  Run-Roll-Aim  test,  were  selected  for 
the  first  analysis.  This  first  analysis  was  confined  to  the  “roll”  section  of  the  test.  Each  subject’s  performance  on 
the  test  was  recorded  with  a  3-axial  accelerometer  and  a  3-axial  gyroscope  (NexGen  Ergonimics  SXTIMU  8GB 
memory)  attached  to  the  head  and  another  such  pair  attached  to  the  torso.  Thus,  the  data  obtained  from  each 
subject  comprise  12  time  series  of  measurements.  Each  time  series  was  transformed  into  the  frequency  domain 
using  512-point  Fast  Fourier  Transform  (FFT),  yielding  256  data  points  for  each  time  series  and  the  total  of  12288 
data  points  from  12  sensors  and  4  test  repeats  (256x4x12).  Out  of  these  12288  data  points,  300  points  were 
selected  as  discriminative  features. 

To  visualize  the  distribution  of  37  HC  and  37  mTBI  subjects  in  such  300-dimensional  feature  space  (in 
which  each  subject  is  represented  by  a  point),  all  74  subjects  were  projected  onto  a  2-dimensional  plane,  which 
was  chosen  by  PLS-DA  (Partial  Least  Squares  -  Discriminative  Analysis)  algorithm  so  as  to  maximize  the  spatial 
separation  of  HC  and  mTBI  (labeled  “BI”  on  Figures  IVA-IVC)  subjects.  This  projection  is  shown  in  Figure  2, 
revealing  prominent  and  distinct  clustering  of  the  two  groups. 
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Figure  2.  Combat  Roll:  Principal  Component  Analysis  of  mTBI  (“BI”)  and  HC 
subjects 
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Next,  a  linear  Support  Vector  Machine  (SVM)  was  trained  to  discriminate  between  HC  and  mTBI 
subjects  using  their  300-feature  vectors  as  inputs.  The  discriminative  performance  of  SVM  was  cross-validated 
using  the  leave-one-out  approach.  31  out  of  37  mTBI  subjects  (84%)  were  correctly  classified  by  SVM  as  such, 
and  23  out  of  37  HC  subjects  (62%)  were  also  classified  correctly.  The  distributions  of  SVM  scores  of  BI  and  HC 
populations  are  plotted  in  Figure  3,  showing  significant  shift  of  the  mTBI  (“BI”)  distribution  relative  to  the  HC 
distribution. 


Figure  3.  SVM  discrimination  of  mTBI  (“BI”)  versus  HC  subjects 


Figure  4.  The  ROC  curve  of  the  mTBI  (“BI”)  and  HC  distributions.  Area  under  the  ROC 
curve  (AUC)  is  0.804. 
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The  ROC  curve  of  the  mTBI  and  HC  distributions  is  plotted  in  Figure  IVC.  Area  under  the  ROC  curve  (AUC)  is 
0.804  (Figure  4). 

All  of  the  above  results  were  obtained  only  from  roll  segments.  When  we  analyzed  other  Run-Roll- Aim  test 
sections  (i.e.,  forward  run  and  backward  run),  we  found  AUC  =  0.554  for  forward  run  and  AUC  =  0.582  for 
backward  run.  When  we  merged  forward,  roll  and  backward  sections  together,  we  found  AUC  =  0.585.  These 
results  show  that  the  roll  section  contains  much  more  information  to  discriminate  subjects  with  mTBI  than  the 
other  sections  of  the  task. 

Discussion 

The  Run-Roll-Aim  task  includes  complex  activities  that  are  performed  sequentially  with  the  entire  task 
taking  less  than  a  minute  to  complete  in  most  instances.  There  are  challenges  with  the  fast  paced  performance  of 
the  task  for  the  examiner  who  operates  the  slides  that  cue  each  part  of  the  task  to  be  able  to  watch  performance  of 
the  task,  and  record  responses  all  at  the  same  time.  Further  examination  of  the  type  of  errors  that  were  made 
during  testing  may  be  instructive  in  understanding  if  the  directional  Stroop  cue  should  continue  to  be  a  part  of 
future  testing.  It  is  likely  that  more  sensitive  instrumentation  may  be  required  to  detect  subtle  delays  associated 
with  information  processing  during  the  task. 

The  visual  stimuli  that  were  used  in  the  task  are  relatively  crude  (numbers  on  an  array  on  the  computer 
screen).  In  addition  it  was  not  uncommon  for  subjects  to  experience  some  difficulty  with  viewing  the  visual 
targets  via  the  close  range  short  focal  point  scope,  as  a  small  shift  in  position  could  cause  the  scope  to  require 
refocusing.  The  visual  aspect  of  the  task,  if  it  is  to  be  continued  could  be  improved.  Virtually  no  one  had  difficulty 
avoiding  the  trip  wire,  so  this  aspect  of  the  task  could  be  modified  or  eliminated.  It  is  clear  that  some  subjects  with 
mTBI  had  symptoms  that  were  provoked  such  that  they  couldn’t  complete  all  the  trials  of  the  task,  so  including 
some  higher  level  mobility  skill  such  as  this  in  future  test  batteries  is  advised. 

In  order  to  capture  movement  differences  between  groups,  the  inertial  sensors  were  essential.  It  is 
possible  that  the  task  could  be  further  instrumented  to  improve  the  ability  to  detect  performance  issues  that  are 
difficult  to  observe.  The  face  validity  of  the  task  can’t  be  denied,  however,  so  the  potential  for  this  task  to  be 
refined  further,  perhaps  using  a  shorter  distance  to  make  it  more  clinically  feasible  is  reasonable  to  pursue  in 
future  testing. 

Recommendations 

As  described  in  the  discussion  section  above,  a  number  of  revisions  to  this  task  would  be  required  to 
improve  the  observational  metrics  before  it  might  be  considered  for  use  to  discriminate  HCs  from  SM  with 
residual  mTBI  issues.  It  appears  that  the  most  salient  components  of  this  task  are  the  symptoms  evoked  and 
pattern  of  movement  detected  by  the  combat  rolls  which  may  only  be  detected  using  very  sophisticated  time  series 
analyses  of  inertial  sensor  data.  Possible  revisions  to  the  observational  metrics  of  this  task  may  involve  further 
instrumentation  to  a  mock  weapon  to  allow  detection  of  the  time  it  takes  a  subject  to  acquire  a  target  after  one  or 
several  combat  rolls  in  order  to  capture  the  effects  of  vestibular  vulnerabilities  following  mTBI.  There  are  a 
number  of  additional  analyses  that  are  to  be  completed  for  the  current  inertial  sensor  data  set  including  further 
standard  analyses  on  angular  accelerations,  and  head  and  torso  path  during  the  combat  rolls. 


ILLINOIS  AGILITY  TEST-PACKING  LIST 

Introduction 

The  Illinois  Agility  Test-Packing  List  is  a  standard  dual-task  with  the  expectation  that  there  would 
be  a  dual-task  cost  in  terms  of  number  of  recalled  packing  list  words  cost  in  terms  of  time  to  run  the  course. 

Description 
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The  Illinois  Agility  Test-Packing  List  requires  running  distances  of  30’  with  rapid  direction  changes  and 
navigation  of  serpentine  or  figure  of  eight  obstacles  during  the  middle  part  of  the  obstacle  course20.  Next,  a 
packing  list  memory  task  assesses  the  participant’s  ability  to  retain  up  to  7  items  on  a  word  list.  The  results  of  this 
single  condition  memory  task  are  used  as  a  baseline  for  the  dual-task  condition.  The  agility  task  and  the  memory 
task  are  done  at  the  same  time  (i.e.,  in  a  dual-task  condition).  Accuracy  of  memory  recall  and  time  to  complete  the 
agility  task  are  measured  in  single  and  dual-task  conditions.  Running  the  agility  course  was  selected  for  dual-task 
performance  because  walking  may  be  too  simple  for  SMs  who  have  high  levels  of  athletic  ability.  The  ability  to 
move  quickly  while  remembering  information  has  relevance  to  duty  situations.  This  test  takes  approximately  12 
minutes  to  perform. 

Rationale 

Given  another  AMMP  task  (ISAW-Grid)  used  a  number  memory  task,  comparable  to  a  task  from  the 
Walking  And  Remembering  test,  during  the  Illinois  Agility-Packing  List  task  we  devised  an  alternative  word  list 
task  (packing  list)  in  order  to  increase  the  diversity  of  cognitive  challenges  in  the  test  battery.  A  grocery  list 
shopping  task  developed  by  Hyndman21  for  use  in  stroke  survivors  was  used  initially  in  pilot  testing  with  ROTC 
cadets  at  University  of  North  Carolina-Chapel  Hill  (UNC-CH).  The  “face  validity”  of  a  grocery  list  was  lacking, 
however,  so  we  worked  with  our  military  team  members  to  create  lists  of  items  that  would  be  appropriate  to  pack 
if  a  service  member  were  to  be  deployed.  We  included  clothing/personal  gear,  weapons,  and  first  aid  on  each  7 
item  list,  making  the  total  number  of  syllables  in  each  list  comparable.  Through  initial  testing  we  identified  items 
that  were  commonly  confused  when  recited  out  loud,  and  simplified  many  of  the  items  included  on  each  list.  We 
also  made  sure  that  items  that  could  easily  be  associated  with  each  other  were  not  included  on  the  same  word  list, 
and  created  6  words  lists  for  use  during  testing.  No  items  were  repeated  on  any  list. 

This  cognitive  task  is  a  working  memory  task,  and  we  know  that  typical  working  memory  span  is  7  +/-  2. 
Consistent  with  testing  that  has  been  done  with  individuals  with  more  moderate  TBI  and  with  older  adults,  we 
modified  the  word  list  length  for  each  subject  based  on  how  they  performed  on  the  first  word  list  task,  therefore 
the  cognitive  task  could  include  5  to  7  words,  presenting  a  cognitive  challenge  that  is  “customized”  to  each 
subject.  This  task  is  more  difficult  than  number  list  memory  tasks,  because  the  word  lists  cannot  be  as  easily 
“chunked”  for  memory  as  numbers  can  (1-3-4  can  be  remembered  as  134  and  reduce  the  working  memory  load 
from  3  items  to  1). 

Given  dual-task  researchers  have  advocated  for  priority  instructions  during  repeated  dual -task  practice, 
we  had  individuals  perform  the  first  dual-task  trial  as  they  would  do  spontaneously  and  then  asked  subjects  to 
prioritize  the  cognitive  or  motor  task  in  repeated  trials  (counterbalanced).  The  theory  for  including  these 
instructions  relates  to  the  ability  to  flexibly  allocate  attention  to  one  task  or  the  other,  a  volitional  prioritization  we 
would  expect  an  HC  subject  to  accomplish,  but  perhaps  those  with  mTBI  may  have  more  difficulty  completing. 

Since  dual-task  movement  impairments  post-concussion  have  only  been  detected  using  laboratory 
measures,  we  incorporated  the  use  of  wireless  inertial  sensors  (see  description  under  Run-Roll-Aim  task)  on  the 
head  and  torso  so  that  we  might  compare  more  instrumented  measures  between  groups  in  the  event  that  the  simple 
observational  measures  did  not  detect  differences. 

Hypotheses/Expectations 

We  anticipated  that  SM  would  be  motivated  to  run  the  agility  course  quickly.  We  expected  to  see  dual¬ 
task  interference  during  the  dual-task  conditions  that  would  likely  affect  the  cognitive  task  in  the  uninstructed 
condition,  given  SM  place  significant  value  on  physical  skills.  During  the  prioritized  conditions  we  expected  that 
greater  dual-task  interference  would  be  observed  on  the  cognitive  task  in  the  motor  priority  task,  and  on  the  motor 
task  in  the  cognitive  priority  task.  We  expected  that  we  could  see  some  issues  with  fatigue  toward  the  end  of  the 
trials  so  we  planned  to  provide  brief  rests  especially  if  a  subject  demonstrated  increased  respiratory  rate.  Since 
this  is  a  new  task  combination,  we  did  not  know  what  the  amount  of  dual-task  interference  would  be  that  is 
“normal”,  but  expected  that  those  with  mTBI  would  show  greater  dual-task  interference  than  the  healthy  controls 
across  conditions. 
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We  expected  that  individuals  with  vestibular  impairment  as  a  result  of  mTBI  may  have  difficulty  with  the 
transitions  from  prone  to  standing  and  with  the  turns  necessary  to  traverse  the  obstacle  course.  In  addition  we 
anticipated  that  individuals  with  mTBI  could  have  difficulty  completing  this  running  task  as  a  result  of  balance 
impairment  or  as  a  result  of  difficulty  with  exertional  symptoms. 

Method 

As  part  of  the  AMMP  validation  study  to  evaluate  inter-rater  reliability  and  known-group  discrimination, 
the  Run-Roll-Aim  was  administered  to  SM  with  mTBI  and  HC.  Observational  and  inertial  sensor  data  were 
collected.  Between-groups  analyses  for  run  time  and  recalled  packing  list  words  in  single  and  dual-task  conditions 
were  conducted  on  50  HC  and  45  SM  with  mTBI. 


Results 


Observational  data 

Acceptable  IRR  of  observed  responses  were  obtained  during  Illinois  Agility-Packing  List  testing  with  HC 
and  those  with  mTBI,  although  some  work  was  required  to  improve  rules  for  interpreting  correct  word  responses 
(only  accepting  exact  responses  as  correct  and  removing  multiple  part  words  from  the  packing  lists,  i.e.,  100  mph 
tape). 

The  Illinois  Agility-Packing  List  dual-task  condition  was  not  effective  in  differentiating  between  HC  and 
mTBI  subjects  when  comparing  single  or  dual-task  performance  scores  based  on  observation  (words  correct  and 
movement  speed)  (Table  IVE(l)).  Group  differences  in  movement  were  detected  with  instrumented  inertial 
sensors,  consistent  with  previous  research  on  concussion. 


Table  IVE(l).  Illinois  Agility-Packing  List  Between  Groups  Comparisons  and  Correlations 


Scoring 

Reliability 

Known  Groups  Analysis* 

Correlations  to  Standard 
Neurocognitive  tests 

Metrics 

Fort  Bragg 

HC 

mTBI 

ICC  (95%  Cl),  n,  p-value 

n  =  23 

18  mTBI, 

5  HC 

n  =  50 

Mean  (SD) 
Median  (Range) 

n  =  45 

Mean  (SD) 
Median  (Range) 

p-value 

ICC 

(95%  Cl) 

TOH,  number  of 

moves 

WRAT  Reading 

(IQ) 

Words 

correct 

Single  task 

1.0 

(1-1) 

5. 5(0. 8) 
6(4-7) 

5. 6(0.8) 
6(4-7) 

0.722 

-0.13  (-0.33,0.08) 
n=86  p=0.235 

0.32  (0.12,  0.49) 
n=95,  p=0.002* 

Time(secs) 
Single  task 

0.99 

(0.98-0.99) 

19.8(2.3) 

19.1(16.1-26.1) 

19.9(2.3) 

19.1(16.1-25.0) 

0.338 

0.08  (-0.13,0.29) 
n=86  p=0.467 

0.01  (-0.19,  0.21) 
n=95  p=0.911 

Word  errors 
Single  task 

1.0 

(1-1) 

1.7(1. 2) 
2(0-5) 

1.6(1. 2) 
1(0-5) 

0.483 

0.16  (-0.05,  0.36) 
n=86  p=0.141 

-0.3  (-0.48,  -0.11) 
n=95,  p=0.003* 

Words 

correct 

Dual- 

Cognitive 

1.0 

(1-1) 

4. 5(1.0) 
5(2-6) 

4. 3(1.0) 
4(2-6) 

0.354 

-0.05  (-0.26,  0.16) 
n=86  p=0.646 

0.4  (0.22,  0.56) 
n=95,  p<0.001* 

Time  (secs) 

Dual- 

Cognitive 

0.99 

(0.989-.997) 

18.8(1.9) 

18.4(15.5-23.9) 

19.3(2.2) 

18.8(15.2-26.0) 

0.320 

0.23  (0.02,  0.42) 
n=86,  p=0.034* 

0.11  (-0.09,  0.31) 
n=95  p=0.268 

Word  errors 

Dual- 

Cognitive 

0.996 

(0.987-1) 

1.3(1. 3) 
1(0-4) 

1.6(1. 3) 
1(0-5) 

0.387 

-0.03  (-0.24,  0.19) 

n=86  p=0.808 

-0.22  (-0.4,  -0.02) 
n=95,  p=0.031* 

*  Known  groups  analysis  with  Mann-Whitney  U  unless  otherwise  noted 
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We  also  discovered  that  subjects  were  not  shifting  priority  to  the  cognitive  task  even  if  instructed  to  do  so. 
This  was  the  case  for  those  with  or  without  mTBI.  Our  efforts  to  compare  dual-task  costs,  as  is  the  customary 
approach  was  confounded  by  our  use  of  different  length  word  lists  based  on  the  single  task  testing.  In  these  tests 
when  there  are  multiple  trials  of  cognitive  tasks,  there  is  the  potential  for  recalling  words  from  prior  word  lists  in 
the  later  trials.  Conversely,  there  is  the  potential  for  reduction  in  movement  speed  in  later  agility  test  trials  as  a 
result  of  fatigue.  The  results  below  show  that  a  greater  number  of  mTBI  subjects  were  not  able  to  complete  the 
series  of  trials,  a  result  approaching  significance  (Table  IVE(2)). 


Table  IVE(2).  Illinois  Agil 

ity-Packing  List  Dual  Task  Cost  Findings 

Metric  (50HC,  46mTBI) 

HC 

Mean(SD) 

Median(Range) 

mTBI 

Mean(SD) 

Median(Range) 

p-value,  Method 

Single  word 

5. 5(0.8) 

5. 5(4-7) 

5. 6(0.8) 

6(4-7) 

0.396,  Mann-Whitney 

Single  Time(secs) 

19.6(2.4) 

19.3(16.1-26.1) 

19.8(2.3) 

19.1(16.1-25.0) 

0.514,  Mann-Whitney 

Single_  W  ord_error 

1.8(1. 2) 

2(0-5) 

1.6(1. 2) 

1(0-5) 

0.259,  Mann-Whitney 

Cost  dni  time  (%) 

-2.6(4. 8) 
-1.7(-16.9-7.6) 

-2. 8(9.0) 

-1.6(-28. 2-22.1) 

0.970,  Mann-Whitney 

Cost  mob  time(%) 

2. 9(5.4) 
2.9(-7.4-19.1) 

2.4(8. 5) 
2.8(-28.2-21.2) 

0.529,  t-Test 

Cost  dcog  time(%) 

-1. 6(7.4) 
-1.7(-15.3-17.2) 

-2.8(12.2) 

-1.1  (-38. 1-28.9) 

0.837,  Mann-Whitney 

Cost  dni  word(%) 

18.9(16.9) 

16.7(0-66.7) 

18.7(16.1) 

18.3(0-66.7) 

0.970,  Mann-Whitney 

Cost  mob  word(%) 

20.8(18.2) 

20.0(-25.0-50.0) 

23.0(16.3) 

20.0(0-66.7) 

0.575,  Mann-Whitney 

Cost  dcog  word(%) 

17.2(16.6) 

16.7(0-50.0) 

23.3(16.9) 

20.0(0-60.0) 

0.075,  Mann-Whitney 

Cost  dni  word  errors(%) 

31.1(113.1) 

45.0(-300-300) 

33.2(117.2) 

50.0(-300-300) 

0.932,  Mann-Whitney 

Cost  mob  word 
errors(%) 

9.9(137.4) 

167(-300-400) 

31.4(139.9) 

50.0(-400-400) 

0.300,  Mann-Whitney 

Cost  dcog  word 
errors(%) 

37.3(110.0) 

50(-200-400) 

35.4(142.8) 

50(-300-400) 

0.793,  Mann-Whitney 

Complete  all  trials 

50  of  53(94.3%) 

46  of  55(83.6%) 

0.070,  Fisher  Exact 

DNI  =  do  not  instruct;  MOB 

=  Mobility  priority;  COG  = 

cognitive  priority 

Inertial  sensory  data 

Inertial  sensor  data  analysis  steps  are  illustrated  in  the  Figure  5  (described  below). 
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Figure  5.  Analysis  steps  for  Illinois  Agility-Packing  List  task  inertial  sensor  accelerometer  data. 

Only  the  accelerometer  trace  from  the  NexGen  sensors  were  used  for  this  analysis.  Linear  data  from  the  3 
axes  of  accelerometry  and  one  non-linear  measure  h  were  included  in  this  first  attempt  to  analyze  the  inertial 
sensor  data.  There  were  18  HC  and  18  mTBI  subjects  included  in  the  analysis.  Two  principal  components  were 
extracted  as  a  way  to  reduce  the  amount  and  redundancy  of  available  data  to  be  considered.  The  data  were  then 
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Fast  Fourier  Transformed  to  examine  the  frequencies  in  the  movements  as  the  next  step.  Most  frequencies  were  in 
the  lower  domains,  so  the  190  frequencies  with  the  highest  level  of  power  were  used  in  the  creation  of  a  linear 
support  vector  machine.  This  method  is  commonly  used  in  classification  research.  The  SVM  was  trained  with  the 
data  from  the  mTBI  and  HC  subjects  considering  the  differences  in  the  190  characteristics  of  movement  during 
the  Illinois  Agility-Packing  List  task.  The  ability  of  the  SVM  to  correctly  classify  was  then  cross  validated  using 
the  leave  one  out  approach,  where  each  subject  was  assigned  a  score  that  was  <  0  if  HC  and  >0  if  in  the  mTBI 
group.  The  results  of  the  classification  showed  14/18  control  subjects  (77.7%  accuracy)  were  correctly  classified 
and  17/18  mTBI  subjects  (94.4%  accuracy).  The  results  from  the  ROC  curve  are  shown  below,  where  the  area 
under  the  curve  is  .807  (Figure  6). 
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Figure  6.  ROC  curve  for  Illinois  Agility-Packing  List  using  SVM  classification 


The  Principal  Components  Analysis  done  in  the  process  of  preparing  the  data  for  the  SVM  can  also  be  used  to 
visualize  potential  differences  in  movement  characteristics  between  the  groups,  demonstrating  separation  of  the 
groups  in  a  way  that  is  very  encouraging  given  this  first  attempt  at  analysis  of  the  sensor  data  (Figure  7). 
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Figure  7.  Principal  Component  Analysis  to  discriminate  mTBI  and  HC  subjects  in  Illinois  Agility -Packing  List 
prone  to  stand  component. 

Discussion 

Unfortunately,  the  expected  dual-task  interference  for  the  cognitive  task  (seen  in  other  dual-task  items  in 
the  AMMP)  was  not  observed  with  observational  measures.  Active  duty  SM  are  continually  trained  on  agility 
type  tasks  and  SMs  are  very  competitive,  so  there  were  not  group  differences  in  time  for  those  who  could 
complete  the  task.  There  were  subjects  with  mTBI  and  one  HC  subject  (aggravation  of  prior  minor  ankle 
condition)  who  were  not  able  to  complete  all  the  trials,  so  for  those  individuals,  this  task  did  identify  problems 
with  higher  level  exertion  with  quick  running. 

The  cognitive  task  used  in  this  task  was  problematic.  We  allowed  a  shorter  word  list  for  those  with 
limited  working  memory.  Post-study  advisement  by  a  neuropsychologist  suggests  that  using  the  7  item  word  list 
for  all  subjects  may  be  a  better  strategy  in  future  testing.  The  lack  of  range  that  was  introduced  by  making  the  task 
easier  for  subjects  with  shorter  working  memory  span  may  have  reduced  the  ability  to  identify  deficits,  and  also 
reduce  the  possible  dual-task  costs  given  those  values  are  computed  based  on  single  task  performance.  Likely 
requiring  this  longer  word  list  for  everyone  would  challenge  those  with  mTBI  to  a  greater  degree  and  illustrate 
dual-task  effects.  For  future  testing,  either  use  of  a  consistent  7  item  word  list  or  consideration  of  the  grid  memory 
task  may  be  advisable.  Given  the  number  memory  task  is  one  that  has  been  tested  more  consistently  through  the 
Walking  and  Remembering  Test22,  and  demonstrated  group  differences  in  the  current  study,  considering  the 
flexible  use  of  mobility  tasks  paired  with  differing  cognitive  tasks  may  be  studied  further.  Our  efforts  to  have 
subjects  “shift  priority”  to  the  cognitive  task  were  not  successful.  This  is  not  completely  surprising,  given  the 
value  that  individuals  in  the  military  place  on  physical  prowess.  For  future  testing  in  similar  groups,  use  of  the 
priority  instructions  may  be  eliminated. 

Recommendations 
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Revision  of  the  cognitive  component  of  this  task  and  further  re-evaluation  of  this  tasks  ability  to 
discriminate  groups  is  recommended.  This  is  a  higher  level  mobility  task  and  with  an  appropriate  and  more 
challenging  cognitive  task  might  be  expected  to  distinguish  mTBI  with  residual  issues  and  HC  subjects  at  the 
point  they  are  thought  to  be  ready  to  return  to  duty.  There  are  a  number  of  additional  analyses  that  are  to  be 
completed  for  the  current  inertial  sensor  data  set  including  further  standard  analyses  on  angular  accelerations  and 
path  specifically  during  the  serpentine  partial  turns  in  this  task.  Once  this  is  done,  further  exploration  of  the  use 
of  time  series  analyses  with  inertial  sensor  data  on  a  modified  Illinois  Agility  task  should  be  considered. 


Introduction 


INSTRUMENTED  STAND  AND  WALK  -  GRID  COORDINATES 


The  ISAW-Grid  task  involves  using  wireless  wearable  inertial  sensors  (Opal  System  includes  precision 
triaxial  accelerometers,  gyroscopes  and  magnetometers)  and  a  clinical  software  program  to  measure  static  postural 
sway  and  then  dynamic  stability  during  walking  and  turning.  A  grid  memorization  task  provided  in  the  context  of 
a  patrol  mission  provides  the  cognitive  challenge.  This  task  may  be  better  tolerated  all  across  the  recovery 
continuum  when  other  AMMP  tasks  may  be  too  challenging  for  the  acutely  concussed  soldier.  Walk  time  and  the 
number  of  recalled  grid  coordinates  are  the  metrics  along  with  the  sensor  data. 

Description 

The  SM  performs  the  Instrumented  Stand  and  Walk  (ISAW)  test  (developed  by  APDM,  Inc.,  Portland, 
OR)  which  includes  instrumented  and  timed  assessment  of  quiet  standing  for  30  seconds,  assessment  of  dynamic 
stability  during  walking  for  two  7  m  (23  foot)  lengths  with  a  180  degree  turn  at  midpoint  (Mancini  et  al  2012)  - 
see  Figure  8.  The  SM  was  asked  to  memorize  an  8  digit  alphanumeric  grid  coordinate  provided  within  the  context 
of  a  simulated  patrol  mission  brief  and  report  the  exact  sequence  back  to  the  examiner  after  45  seconds.  Finally, 
both  the  ISAW  and  the  grid  memorization  tasks  will  be  performed  simultaneously.  Accuracy  of  grid  coordinate 
recall,  and  a  number  of  inertial  sensor  variables  including  postural  sway  area,  gait  path  variability,  turn  dynamics 
and  time  to  complete  the  ISAW  (i.e.  gait  speed)  were  measured  in  single  and  dual-task  conditions.  This  test  takes 
approximately  15-17  minutes  to  perform  including  3  trials  of  the  single  task  and  3  trials  of  the  dual-task  condition 
in  addition  to  instructions  and  sensor  set-up. 


-V 


30  Seconds 


tAlAMA> 


Quiet  Stance  |- 


Walk  7  meters 
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Figure  8.  The  ISAW  protocol,  courtesy  of  APDM,  Inc.,  Portland,  OR. 


Rationale 

This  task  assesses  balance  during  normal  standing  and  gait  stability  when  walking  at  a  comfortable  pace 
and  including  a  180  degree  turn  as  well  as  working  memory  under  normal,  sub-maximal  exertion  conditions23. 
The  ability  to  learn  and  retain  operationally  relevant  information  such  as  that  provided  in  this  task  while 
completing  a  motor  task  has  relevance  to  functional  duty  demands.  It  is  also  not  a  strenuous  physical  task  and 
would  be  expected  to  be  tolerated  by  subjects  with  acute  concussion. 


Hypotheses/expectations 
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A  dual-task  paradigm  requires  an  individual  to  perform  two  tasks  simultaneously  (in  this  project:  a  motor 
and  a  cognitive  task),  in  order  to  compare  dual-task  performance  with  single-task  performance.  The  interpretation 
assumes  that  human  processing  resources  are  limited  and  shareable;  and  that  in  concussion  and  other  types  of 
mild  brain  injury  an  overload  in  these  processing  resources  is  more  easily  reached.  Dual-task  performance  is 
often  measured  as  a  “cost”  relative  to  performance  of  each  activity  in  a  single  domain  condition.  Dual-task  effects 
include  a)  interference  (cost)  which  is  defined  as  performance  under  dual-task  conditions  results  in  a  decrement 
relative  to  single  task  performance.  A  second  dual-task  effect  is  called  b)  enhancement  (benefit)  in  which 
performance  under  dual-task  conditions  results  in  an  improvement  relative  to  single  task  performance.  Gait  and 
executive  functioning  (cognitive  process  that  regulates,  controls,  and  manages  other  cognitive  processes; 
executive  functioning  includes  ability  to  divide  attention  during  ambulation)  become  more  closely  associated  as 
locomotor  challenge  becomes  more  challenging  and  with  the  addition  of  a  dual-task  such  as  walking  while 
reading  signs  or  while  talking  with  a  friend. 

The  ISAW-Grid  is  a  standard  dual-task  with  the  expectation  that  there  would  be  a  dual-task  cost  in  terms 
of  number  of  recalled  grid  coordinates,  the  time  to  walk  the  course  in  addition  to  inertial  sensor  data  findings  with 
a  special  focus  on  sway  area,  gait  velocity  and  variability  and  turn  dynamics.  We  also  in  general  expected  that 
subjects  with  mTBI  would  have  more  difficulty  with  the  grid-coordinate  recall  in  both  the  single  and  the  dual-task 
condition  due  to  issues  with  memory  and  to  a  lesser  extent,  attention.  A  correlation  to  the  standard  memory 
neurocognitive  test  of  NAB  Digits  forward/Digits  backward  and  NAB  Numbers  and  Letters  was  expected. 

Method 


As  part  of  the  AMMP  validation  study  to  evaluate  IRR,  known-group  discrimination  and  convergent  and 
discriminate  validity,  the  ISAW-Grid  was  administered  to  SM  with  mTBI  and  HC  subjects.  ISAW-Grid 
assessment  relied  on  observational  metrics,  including  number  of  recalled  grid  coordinates  and  time  (measured 
with  a  stopwatch)  to  complete  the  7  meter  walk  with  the  180  degree  turn  at  the  7  meter  mark.  Inertial  sensor  data 
as  described  above  was  also  collected  using  a  commercially  available  software  program  (ISAW-Grid)  from 
APDM,  Inc.  (Portland,  OR). 

Between-groups  analyses  for  recalled  grid  coordinates  and  for  walk  time  in  single  and  dual-task  condition 
were  conducted  on  52  HCs  and  51  SM  with  mTBI.  Between-groups  analyses  for  all  inertial  sensor  variables 
(www.apdm.com/mobility)  were  also  evaluated  for  42  subjects  with  mTBI  and  62  HCs.  NOTE  that  the  inertial 
sensor  findings  were  evaluated  on  HC  subjects  both  at  USARIEM  and  at  Fort  Bragg.  A  comparison  of  data  was 
done  between  20  HCs  from  USARIEM  and  HCs  from  Fort  Bragg  with  no  differences  found  for  the  variables 
related  to  turn  metrics  (p  =  0.1 1  to  0.78),  which  thus  allowed  the  data  to  be  combined. 

Inertial  sensor  data 

Our  approach  used  wireless  sensors  on  a  belt  placed  at  the  lumbar  spine  (Opal)  and  around  the  ankles. 
Each  sensor  has  a  3D  accelerometer  and  3D  angular  rate  sensor,  and  uses  a  Mobility  Lab  (APDM  company, 
Portland  OR)  system  to  automatically  quantify  sway,  gait,  and  turning  metrics.  Data  from  these  sensors  is 
wirelessly  transmitted  (via  radio  waves)  to  a  laptop  where  automatic  algorithms  calculate  up  to  42  different  sway 
and  gait  measures  for  each  trial24.  Using  the  commercially  available  IS  AW  program,  3  trials  of  single  task  stand 
and  walk  were  collected.  The  median  scores  were  used  for  the  between  groups  analysis;  this  was  also  done  in  the 
dual-task  condition.  The  primary  variables  that  we  planned  evaluate  included: 

Recorded  and  calculated  by  APDM  Opal  system.  To  include,  but  not  limited  to: 

o  Centroidal  Frequency  during  sway 

o  Frequency  dispersion  during  sway 

o  Median  RMS  Postural  Sway  (total  sway  area,  cm2)  in  single  task  condition 

o  Median  RMS  Postural  Sway  (total  sway  area,  cm2)  in  dual-task  condition 

o  Gait  Speed  (m/s)  in  single  task  condition 
o  Gait  Speed  (m/s)  in  dual-task  condition 
o  Gait  variability  (CV)  in  single  task  condition 
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o  Gait  variability  (CV)  in  dual-task  condition 
o  Peak  Turning  velocity  in  single  task  condition 
o  Peak  Turning  velocity  in  dual-task  condition 


Results 

Observational  data 

Number  of  recalled  grid  coordinates  in  the  dual-task  condition  was  able  to  distinguish  groups  (p<0.008). 
Recall  of  grid  coordinates  in  the  single  task  condition,  as  well  as  walk  time  (seconds)  in  the  single  and  dual -task 
condition  did  not  distinguish  groups  (Table  IVF(l)).  Findings  presented  as  dual  task  cost  are  presented  in  Table 
P/F.  Our  findings  appear  to  indicate  that  the  calculation  of  cost  is  less  informative  in  this  group  of  physically 
elite  subjects  than  is  often  described  in  the  literature  on  dual  task  interference  in  the  elderly  or  those  with  mild 
cognitive  impairment.25 

Inertial  sensor  data 

Demographics  for  subjects  tested  with  inertial  sensors  for  the  ISAW-Grid  task  are  found  in  Table  IVF(2). 
Instrumented  measures  of  turning  revealed  significant  between  group  differences  (Table  IVF(3)).  In  both  single 
and  dual-task  conditions,  soldiers  with  post-concussive  deficits  demonstrated  longer  turn  durations,  increased  step 
numbers  to  complete  a  turn,  and  decreased  peak  rotational  velocities  during  turns.  All  of  these  turning  variables 
are  correlated  to  each  other  and  calculated  from  the  same  data,  thus  would  be  expected  to  all  distinguish. 

Variables  related  to  gait  velocity,  step  variability,  centroid  frequency  and  total  sway  area  did  not  distinguish 
groups  (p>0.10  for  all  variables). 

Table  IVF(l).  Instrumented  Stand  and  Walk  -  Grid  Coordinates  (ISAW-Grid) 


Scoring 

Reliability 

Known 

Groups 

Analysis* 

Correlations  to  Standard  Neurocognitive  tests 

Metrics 

Fort  Bragg 

HC 

mTBI 

ICC  (95%  Cl),  n,  p-value 

n  =  26 

19  mTBI, 

7  HC 

N=52 

Mean  (SD) 
Median 
(Range) 

N=51 

Mean  (SD) 
Median 
(Range) 

p-value 

ICC  (95% 

Cl) 

TOH,  #  of 

moves#  moves 

WRAT  Reading  (IQ) 

Grid 

coordinates 

Single 

0.97  (0.92-1) 

6.7(1. 7) 

8  (3-8) 

6.3  (1.8) 

7  (3-8) 

0.262 

-0.19  (-0.38,  0.02) 
n=91  p=0.073 

0.39  (0.21,0.55) 
n=99BMl001#, 

Grid 

Coordinates 

Dual 

0.98  (0.93-1) 

7.0  (1.5) 

8.0  (3-8) 

5.9  (2.2) 
6(1-8) 

|.00| 

-0.09  (-0.29,  0.11) 
n=93  p=0.378 

-0.08  (-0.27,0.11) 
n=103  p=0.416 

Walk  time 
Single 

0.97 

(0.96-0.99) 

12.0  (2.1) 

12.4  (6.9-16.3) 

12.6  (2.2) 
(12.5  8.1-19.0) 

0.206 

t-test 

0.03  (-0.18,0.23) 
n=91  p=0.805 

0.04  (-0.16,  0.24) 
n=99  p=0.697 

Walk  time 
Dual 

0.95 

(0.90-0.98) 

12.0  (2.0) 

12.4  (6.8-15.2) 

12.5  (2.2) 

12.2  (8.5-21.0) 

0.634 

0.04  (-0.17,0.24) 
n=93  p=0.715 

0.14  (-0.05,  0.33) 
n=103  p=0.145 

*Known  groups  analysis  with  Mann-Whitney  U  unless  otherwise  noted 
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Table  IVF(2).  Instrumented  Stand  and  Walk  -  Grid  Coordinates  (ISAW-Grid)-Dual  Task  Costs 


Metric 

HC 

mTBI 

p-value 

Median  dual  grid 
coord(chars)  (52,52) 

7.0(1. 5) 

8.0(3-8) 

5.9(2. 2) 

6. 5(1-8) 

0.016,  Mann- Whitney 

Dual  Task  Cost  grid  (%) 
(52,52) 

-12.6(42.6%) 
0(-166. 67-50. 00) 

4.12(36.8%) 

0(-100. 00-7 1.43) 

0.059,  Mann- Whitney 

Median  dual  time(mins) 
(52,52) 

12.0(2.0) 

12.4(6.8-15.2) 

12.5(5.0) 

12.2(8.5-21.0) 

0.647,  Mann- Whitney 

Dual  Task  Cost  time(%) 
(52,52) 

-0.21(5.4%) 
-0.25(-9. 50-12. 61) 

0.60(5.3%) 

0(-l  1.82-1 1.28) 

0.349,  Mann-Whitney 

Completed  task(52,52) 

100% 

100% 

NA 

Table  IVF(3).  Demographics  for  subjects  tested  with  inertial  sensors  for  ISAW-Grid  task. 


Healthy  Control 

Concussion/mTBI 

P- 

Characteristic 

(N=62) 

(N=42) 

value 

Age  (yrs) 

28.8  (5.5), 

27.1  (5.4), 

(mean  (SD),  median  (range)) 

28.5  (19-42) 

25  (19-42) 

0.052 

Sex-Male 

45  (72.6%) 

39  (92.9%) 

0.011 

Ethnic  Background 

Black 

19  (30.6%) 

3  (7.1%) 

Hispanic 

7  (11.3%) 

9  (21.4%) 

White 

31  (50.0%) 

23  (54.8%) 

Other  (Asian,  Native  American,  biracial) 

5  (8.1%) 

7  (16.7%) 

0.033 

Education 

HS 

12  (19.4%) 

6  (14.3%) 

Post-HS 

22  (35.5%) 

30  (71.4%) 

Bachelors 

19  (30.6%) 

6  (14.3%) 

Post-Grad/Professional 

9  (14.5%) 

0 

0.001 

WRAT  Reading  Level  (mean  (SD),  median 
(range) 

11.9(1.9),  12.7  (4.4-13) 

10.8(2.3),  11.4  (3.8-13) 

0.002 

Deployed  to  Iraq/Afghanistan 

33  (53.2%) 

32  (76.2%) 

0.023 

Sustained  a  concussion 

10(16.1%) 

41  (97.6%) 

0.001 

Years  of  service 

7.0  (5.7),  6.5  (0.3-23.3) 

6.0  (5.1),  3.7  (0.8-23) 

0.619 

Do  you  feel  “Ready  to  Deploy”  to  a  combat 
zone  in  72  hours? 

57  (91.9%) 

20  (47.6%) 

<0.001 

Hearing  issue-self  report 

10(16.1%) 

41  (97.6%) 

<0.001 

#  days  since  last  concussion 

2264(1168.3), 

226.4(140.1), 

N=9,23,  a  lot  of  missing  data 

2023  (977-4030) 

181.5  (58-632) 

0.515 
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Table  IVF(4).  ISAW-Grid  sensor  variables  during  turn  that  distinguished 

i  groups. 

Sensor  Variable 

Median  HC 

Median  mTBI 

p-value 

Peak  Turning  velocity  in  single  task  condition 

235.0  degr/sec 

198.3  degr/sec 

<.001 

Peak  Turning  velocity  in  dual-task  condition 

229.3  degr/sec 

202.0  degr/sec 

<.001 

Turn  duration  seconds  in  single  task  condition 

1.57  sec 

1.88  sec 

<.001 

Turn  duration  seconds  in  dual-task  condition 

1.62  sec 

1.83  sec 

<.001 

Discussion 

Analysis  of  ISAW-Grid  using  the  Mann- Whitney  U  revealed  significant  between  groups 
differences  for  the  grid  task  memory  component  in  the  dual-task  condition  and  not  in  the  single  task 
condition.  This  would  support  the  concept  of  this  dual-task  evaluating  a  limited  and  shareable  amount  of 
brain  resources  as  the  grid  coordinates  distinguished  groups  in  the  dual-task  condition  only.  The  clinical 
measure  of  timed  walk  did  not  distinguish  HC  and  mTBI  groups.  Using  the  inertial  sensor  findings  to 
evaluate  more  subtle  deficits,  only  the  most  dynamic  testing  condition  (a  180  degree  turn)  was  sufficient 
to  discriminate  between  HC  and  SM  with  mTBI  residual  symptoms.  SM  were  premorbidly  high 
functioning  Warrior  athletes;  it  appears  likely  that  this  standing  and  walking  with  a  turn  with  its 
inherently  less  challenging  test  conditions  were  likely  less  sensitive  to  between  group  differences  perhaps 
as  a  result  of  a  ceiling  effect.  Other  studies  use  more  challenging  conditions  such  as  narrowed  stance  or 
gait,  single  leg  stance,  standing  on  a  compliant  surface,  standing  with  vision  suppressed  (ref).  While 
specific  mechanisms  cannot  be  determined  by  the  results  of  this  study,  it  may  be  that  peak  turn  velocity 
may  be  degraded  by  vestibular  symptoms  (e.g.,  diminished  tolerance  for  rotation,  impaired  gaze  or 
dynamic  stability). 

Recommendations 

The  ISAW-Grid  task  may  likely  be  well  tolerated  in  acutely  concussed  SM  and  should  be 
evaluated  in  this  population,  specifically  to  determine  if  it  can  document  recovery  over  time  with  the 
turning  metrics  using  the  inertial  sensor  and  the  cognitive  component  of  number  of  recalled  grid 
coordinates.  As  with  all  AMMP  tasks,  if  this  task  is  to  be  used  to  inform  readiness  to  retum-to-duty,  a 
longitudinal  study  exploring  the  predictive  validity,  information  on  normative  data  on  healthy  controls 
(ultimately  would  want  to  establish  cut-points  for  definite  “Go”  and  “No  Go”  parameters).  If  it  is  to  be 
used  to  measure  progress  for  RTD,  we  need  to  evaluate  responsiveness  to  change  as  a  SM  recovers 
spontaneously  or  responds  to  treatment.  Further  study  is  needed  to  evaluate  the  sensitivity  and  specificity 
of  the  ISAW-grid. 


LOAD  MAGAZINE-RADIO  CHATTER 

Introduction 

In  the  “Load  the  Magazine-Radio  Chatter”  task,  the  subject  loads  M-16  dummy  rounds  from  a  bin 
of  mixed  size  rounds  as  fast  as  possible.  The  dual-task  condition  requires  monitoring  radio 
communication  about  an  upcoming  training  event  and  verbally  announcing  when  key  words  are  spoken 
by  specific  personnel.  Metrics  are  the  number  of  rounds  loaded  and  the  number  of  correct  and  incorrect 
responses  to  the  radio  chatter. 

Description 

The  SM  completes  a  relatively  automatic  manual  task  choosing  from  a  bin  of  mixed  size  dummy 
rounds  (5.56  and  7.62  caliber)  and  loading  5.56  caliber  training  rounds  into  magazines  as  fast  as  possible 
both  in  a  single  and  a  dual-task  condition.  The  dual-task  condition  requires  monitoring  radio 
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communication  and  verbally  announcing  when  specific  key  words  are  spoken  by  specific  personnel.  This 
test  takes  approximately  22-24  minutes  to  perform  including  time  for  set  up  and  subject  instruction. 

Rationale 

The  purpose  of  this  task  is  to  assess  the  cost  of  a  cognitive  task  overlay  requiring  auditory 
processing  on  the  speed  of  a  relatively  automated  upper  extremity  manual  task26.  This  task  is  intended  to 
challenge  attention  allocation  (divided  attention),  sustained  attention,  executive  function,  manual 
dexterity/speed,  and  auditory  processing.  White  noise  in  the  radio  chatter  recordings  require  close 
attention  to  auditory  information  which  is  operationally  important  while  completing  a  motor  task  that  has 
relevance  to  functional  duty  demands.  Given  that  it  is  completed  in  sitting,  this  task  does  not  require 
significant  postural  control. 

Hypotheses/expectations 

A  dual-task  paradigm  requires  an  individual  to  perform  two  tasks  simultaneously  (in  this  project: 
a  motor  and  a  cognitive  task),  in  order  to  compare  dual-task  performance  with  single-task  performance. 
The  interpretation  assumes  that  human  processing  resources  are  limited  and  shareable;  and  that  in 
concussion  and  other  types  of  mild  brain  injury  an  overload  in  these  processing  resources  is  more  easily 
reached.  Dual-task  performance  is  often  measured  as  a  “cost”  relative  to  performance  of  each  activity  in 
a  single  domain  condition.  Dual-task  effects  include  a)  interference  (cost)  which  is  defined  as 
performance  under  dual-task  conditions  results  in  a  decrement  relative  to  single  task  performance.  A 
second  dual-task  effect  is  called  b)  enhancement  (benefit)  in  which  performance  under  dual-task 
conditions  results  in  an  improvement  relative  to  single  task  performance. 

The  Load  the  Magazine-Radio  Chatter  task  is  a  standard  dual-task  with  the  motor  component 
requiring  upper  extremity  function  in  a  relatively  rote  manual  dexterity  task  and  no  real  postural  control 
challenge.  The  expectation  was  for  a  dual-task  cost  in  terms  of  number  of  correct  and  incorrect  key 
words  identified  and  the  number  of  dummy  rounds  loaded  in  the  dual-task  condition  over  the  single  task 
condition.  Given  that  this  task  requires  sustained  attention  with  some  executive  function,  it  was  expected 
to  correlate  to  standard  neurocognitive  tests  of  attention  and  memory  including  the  Comprehensive  Trail 
Making  Test  and  the  NAB  Digits  forward/Digits  backward  and  NAB  Numbers  and  Letters.  A  correlation 
with  WRAT  Reading  was  not  necessarily  hypothesized,  however,  the  expectation  would  be  that  a  subject 
who  is  of  higher  intelligence  may  be  more  able  to  develop  a  strategy  for  being  successful  at  this  task. 

Method 


As  part  of  the  AMMP  validation  study  to  evaluate  inter-rater  reliability,  known-group 
discrimination  and  convergent  and  discriminate  validity,  the  Load  Magazine-Radio  Chatter  task  was 
administered  to  SM  with  mild  traumatic  brain  injury  and  healthy  controls.  Load  Magazine-Radio  Chatter 
task  assessment  relied  on  observational  metrics,  including  number  of  rounds  loaded  (counted)  and  number 
of  key  words  and  distractors  responded  to  in  both  single  and  dual-task  conditions.  Between-groups 
analyses  for  rounds  loaded  and  keywords  and  distractor  words  reported  in  single  and  dual -task  condition 
were  conducted  on  51  HC  and  51  SM  with  mTBI. 

In  order  to  reduce  a  potential  practice  effect,  subjects  are  given  2  timed  trials  to  practice  loading 
dummy  rounds  and  1  to  2  trials  of  listening  to  and  responding  to  radio  chatter  before  being  tested  on  both 
number  of  rounds  loaded  and  number  of  correct  and  distractor  key  words  responded  to  in  both  single  and 
dual-task  conditions. 

Results 

Non  parametric  between  groups  analysis  revealed  significant  differences  in  the  dual-task 
condition  for  the  number  of  correct  responses  to  the  radio  chatter,  and  number  of  errors  in  responding  to 
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distractor  words  committed  between  HC  and  concussed  personnel  (Table  IVG(l)).  The  correct  and 
detractor  responses  to  radio  chatter  keywords  did  not  distinguish  groups  in  the  single  task  condition.  Dual 
task  costs  were  calculated  and  are  are  similar  to  direct  groups  comparisons,  cognitive  task  interference 
demonstrated  in  the  dual  task  condition  (Table  IVG(l)).  The  motor  task  of  loading  rounds  also  did  not 
distinguish  groups  in  either  the  single  or  dual-task  condition.  As  discussed  under  Aim  2,  there  were 
significant  correlations  to  the  CTMT  and  to  the  WRAT  Reading  score  in  the  dual-task  conditions  (Table 
IVG(l)  and  Aim  2,  Table  IIB). 

Discussion 

Subjects  with  mTBI  had  more  difficulty  than  HC  with  the  cognitive  task  (keyword  and  distractor 
word  responses)  specifically  when  the  cognitive  task  was  overlaid  on  the  physical/motor  task  of  loading 
rounds.  This  would  support  the  concept  of  this  dual-task  evaluating  a  limited  and  shareable  amount  of 
brain  resources  as  the  radio  chatter  keyword  responses  distinguished  groups  in  the  dual-task  condition 
only.  It  appears  that  the  motor  task  of  loading  rounds  was  not  of  sufficient  difficulty  to  elicit  a  dual -task 
effect  in  this  Soldier  population,  possibly  because  it  was  a  rote  or  well-trained  task  for  these  subjects  or 
because  subjects  prioritized  motor  task  over  cognitive.  The  Load-Magazine-Radio  Chatter  task  may 
potentially  distinguish  not  duty  ready  SM  with  mTBI  who  have  auditory  processing  issues  including 
those  who  are  irritated  by  loud  sounds. 

Recommendations 

As  with  all  AMMP  tasks,  if  this  task  is  to  be  used  to  inform  readiness  to  return-to-duty,  a 
longitudinal  study  exploring  the  predictive  validity,  information  on  normative  data  on  healthy  controls 
(ultimately  would  want  to  establish  cut-points  for  definite  “Go”  and  “No  Go”  parameters).  If  it  is  to  be 
used  to  measure  progress  for  RTD,  responsiveness  to  change  as  a  SM  recovers  spontaneously  or  responds 
to  treatment  should  be  evaluated.  Further  study  is  needed  to  evaluate  the  sensitivity  and  specificity  of  the 
Load-Magazine-Radio  Chatter  task. 


75 


Table  IVG(l).  Load  Magazine-Radio  Chatter  Between  Groups  Comparisons  and  Correlations 


Scoring 

Reliability 

Known  Groups  Analysis* 

Correlations  to  Standard 
Neurocognitive  tests 

Metrics 

Fort  Bragg 

HC 

mTBI 

ICC  (95%  Cl),  n,  p-value 

n  =  24 

18  mTBI, 

6  HC 

n  =  51 

Mean 

(SD) 

Median 

(Range) 

n  =  51 

Mean 

(SD) 

Median 

(Range) 

p-value 

ICC 

(95%  Cl) 

CTMT 

Composite® 

WRAT  Reading 

(IQ) 

Rounds 
loaded  Single 

Not  evaluated 

55.47 

(9.9) 

58  (32-77) 

53.7  (8.6) 
54  (39-86) 

0.337 

t-test 

* 

* 

Rounds 

loaded  Dual 

Not  evaluated 

51.8(9.3) 
51  (30-72) 

49.1  (9.4) 
49(31-81) 

0.16 

t-test 

* 

* 

Rounds 

difference 

Single-Dual 

Not  evaluated 

3.7  (6.0) 

4  (-13-19) 

4.4  (6.8) 

4  (-13-19) 

0.730 

* 

* 

Single  Task 
Correct 

0.997 

(0.993-1) 

7.7  (1.9) 

8  d-9) 

7.4  (2.3) 

8.5  (0-9) 

0.797 

0.15 

(-0.06,  0.34) 
n=92  p=0.167 

0.15 

(-0.05,  0.33) 
n=102  p=0.139 

Single  Task 
Errors 

0.995 

(0.986-1) 

2.2  (3.4) 

1  (0-17) 

2.7  (3.8) 

1  (0-16) 

0.797 

-0.15 

(-0.34,  0.06) 
n=92  p=0.157 

-0.1 

(-0.29,  0.1) 
n=102  p=0.318 

Dual-task 

Correct 

0.978 

(0.949-0.999) 

7.7  (1.6) 

8  d-9) 

7.02(1.8) 

7  (0-9) 

|.01{ 

0.35 

(0.15,0.52) 
n=92,  p=0.001* 

0.35 

(0.17,  0.51) 
n=102,  p<0.001* 

Dual-task 

Errors 

0.947 

(0.869-1) 

2.2  (2.6) 

2 (0-13) 

3.4  (2.9) 

3  (0-15) 

0.006 

-0.41 

(-0.57,  -0.22) 
n=92,  p<0.001* 

-0.35 

(-0.51,-0.17) 
n=102,  p<0.001* 

Correct 

difference 

Single-Dual 

* 

-0.06  (2.2) 

0  (-6-8) 

0.34  (2.2) 

1  (-7-5) 

0.059 

-0.12 

(-0.32,  0.08) 
n=92  p=0.237 

-0.14 

(-0.32,  0.06) 
n=102  p=0.17 

Rounds 

Cost% 

❖ 

6.1(10.7) 

6.3  (-30.2- 
32.2) 

7.7(12.6) 
7.7(-30.2  - 
35.4) 

0.503 

t-test 

* 

* 

Correct 

Cost% 

-19.4  (92.3) 

0  (-600- 
88.9) 

-14.9 
(117.2) 
ll.l(-700  - 
100) 

|).027| 

* 

* 

*  Known  groups  analysis  with  Mann-Whitney  U  unless  otherwise  noted;  Comprehensive  Trail  Making  Test=CTMT 
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SECTION  V:  EXPLORATORY  ANALYS 

Introduction 

Exploratory  analysis  includes  three  avenues  of  inquiry: 

•  Evaluation  of  participants  who  were  negative  outliers  (did  very  poorly  compared  to  the  rest  of  the 
participants)  on  any  of  the  task  metrics  that  were  able  to  distinguish  between  HC  and  mTBI. 

•  Characteristics  of  subjects  with  mTBI  who  responded  ‘NO’  on  their  intake  form  to  the  question 
about  readiness  to  deploy  on  a  combat  tour  in  72  hours. 

•  Predictive  modeling  to  classify  HC  and  mTBI. 

These  analyses  were  undertaken  in  order  to  further  elucidate  our  findings  from  specific  task  performance 
and  to  identify  opportunities  for  continuing  research. 

OUTLIER  ANALYSIS 

Methods 

Given  the  non-normality  of  the  distribution  of  the  metrics,  we  evaluated  negative  outliers  for  the 
mTBI  group  with  respect  to  the  HC  group  using  the  Tukey  Interquartile  Range  (IQR)  criteria  of  greater 
than  or  less  than  1.5*  IQR  from  either  end  of  the  minimum  /  maximum  edges  of  the  IQR  for  HCs.  Task 
specific  metrics  included  in  this  analysis  were  those  that  were  significantly  different  when  HC  were 
compared  to  mTBI  from  known  group  analysis. 

Results 

A  summary  table  is  provided  (Table  VA).  A  total  of  34  participants  (10HC  and  24  mTBI)  were  outliers 
on  at  least  one  task  item  that  successfully  distinguished  groups. 

CQ  Duty:  For  total  visits,  5  mTBI  were  classified  as  outliers  relevant  to  the  distribution  of  HCs  compared 
to  2  for  Task  Performance.  Total  Visits  appears  to  be  a  more  sensitive  measure  for  identification  of 
outliers. 

ISAW-Grid:  For  number  of  Grid  Coordinates  Correct,  6  mTBI  were  classified  as  outliers  relevant  to  the 
distribution  of  HCs. 

Load  Magazine-Radio  Chatter:  For  these  metrics,  3  HC  and  5  mTBI  were  outliers  on  number  of  keywords 
correct  and  2HC  and  5  mTBI  were  outliers  on  number  of  distractor  errors.  This  task  may  be  less  sensitive 
in  identifying  mTBI  outliers  given  the  number  of  HC  also  identified. 

Patrol-Exertion:  For  these  metrics: 

Vision  end  of  task:  2  HC  and  6  mTBI  respectively  were  outliers 
Reaction  time  -  early:  6HC  and  6  mTBI  were  outliers 
Reaction  time  -  late:  4HC  and  1 1  mTBI  were  outliers 

This  task  may  be  less  sensitive  in  identifying  mTBI  outliers  given  the  number  of  HC  also  identified. 
Reaction  time  late  appears  to  better  distinguish  mTBI  as  outliers  than  early  reaction  time. 

Summary  over  All  Tasks 

Participants  with  greater  than  two  outlier  items:  1HC  and  8  mTBI 
Participant  with  more  than  1  task:  6  mTBI,  4  of  whom  were  not  ready  to  deploy 

Conclusions 
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Another  avenue  for  exploration  for  distinguishing  mTBI  from  HC  as  well  as  readiness  for 
deployment  is  utilization  of  outlier  analysis.  The  mTBI  groups  were  more  likely  to  be  outliers  than  HCs. 
A  large  number  of  service  members  would  need  to  be  tested  in  order  to  establish  norms  and  cut  points  for 
each  of  the  tasks. 

Table  VA.  Negative  outliers  for  the  mTBI  group  with  respect  to  the  HC  group  using  the  Tukey 


Interquartile  Range  (IQR)  criteria 


Outliers 

ID 

CQ  # 

Visits 

CQ  Total 

Performa 

nee 

ISAW# 

Grid  Dual 

Task 

Radio 

Chatter 

#Correct 

Radio 

Chatter 

Errors 

Patrol 

Vision 

Score 

Patrol 

React 

Time 

Early 

Patrol 

React 

Time 

Late 

Total 

Items 

Total 

Tasks 

Group 

Ready 

Deploy 

42 

1 

1 

1 

mTBI 

Y 

51 

1 

1 

1 

mTBI 

N 

57 

1 

1 

1 

mTBI 

N 

67 

1 

1 

1 

3 

2 

mTBI 

N 

78 

1 

1 

1 

mTBI 

N 

79 

1 

1 

1 

3 

1 

mTBI 

N 

85 

1 

1 

1 

mTBI 

Y 

86 

1 

1 

1 

mTBI 

N 

88 

1 

1 

1 

HC 

Y 

92 

1 

1 

1 

mTBI 

N 

93 

1 

1 

1 

mTBI 

Y 

94 

1 

1 

1 

mTBI 

Y 

96 

1 

1 

1 

mTBI 

N 

97 

1 

1 

1 

mTBI 

Y 

99 

1 

1 

1 

HC 

Y 

100 

1 

1 

1 

HC 

Y 

107 

1 

1 

1 

3 

1 

mTBI 

N 

109 

1 

1 

1 

HC 

Y 

114 

1 

1 

1 

1 

4 

2 

mTBI 

N 

116 

1 

1 

1 

mTBI 

N 

117 

1 

1 

2 

1 

HC 

Y 

120 

1 

1 

2 

1 

mTBI 

N 

122 

1 

1 

1 

3 

2 

mTBI 

N 

125 

1 

1 

1 

1 

1 

5 

3 

mTBI 

N 

126 

1 

1 

2 

1 

mTBI 

N 

130 

1 

1 

2 

1 

HC 

Y 

138 

1 

1 

2 

1 

HC 

Y 

143 

1 

1 

1 

3 

1 

HC 

N 

144 

1 

1 

2 

1 

HC 

Y 

152 

1 

1 

1 

mTBI 

Y 

154 

1 

1 

2 

1 

HC 

Y 

164 

1 

1 

1 

3 

2 

mTBI 

Y 

165 

1 

1 

1 

3 

2 

mTBI 

Y 

166 

1 

1 

2 

1 

mTBI 

Y 
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READINESS  TO  DEPLOY 

Methods 

The  Non-Parametric  Wilcoxon  U  Test  was  used  to  compare  those  who  reported  themselves 
‘ready’  or  ‘not  ready’  to  deploy  on  a  combat  tour  in  the  mTBI  group,  on  those  task  metrics  that 
distinguished  groups  as  well  as  neurocognitive  scores  and  relevant  items  on  the  intake  forms.  This  related 
to  a  question  on  the  intake  form  which  asks  subjects  to  provide  a  yes  or  no  response  to  the  question  “Do 
you  feel  mentally  and  physically  prepared  to  begin  a  combat  tour  72  hours  from  now?” 

Results 

Table  VB  describes  those  items  that  distinguished  (p<0.05)  or  nearly  distinguished  the  two  groups 
(p<0.1),  ready  or  not  ready  to  deploy. 

•  Two  task  items  were  significant,  ISAW  grid  number  remembered  under  dual  task  conditions  (p- 
value=  0.045)  and  radio  chatter  number  correct  under  dual  task  conditions  (p-value=  0.029).  Both 
of  these  items  are  related  to  memory  as  is  NAB  Digit  Forward  (p=0.075). 

•  Refer  to  Outlier  Table  V.A,  for  those  participants  with  outliers  in  more  than  one  task;  4  of  6  said 
they  were  ‘NOT  READY  TO  DEPLOY  ON  A  COMBAT  TOUR’. 


Table  VB.  Metrics  that  Distinguish  Groups:  Subject  Reported  “Readiness  to  Deploy  on  a  Combat  Tour” 


ITEM 

Affect  on  'NOT  READY  to  DEPLOY' 

Intake  Form-Sleep  Aide 

Those  using  a  sleep  aide,  more  likely  to  repsond  'NOT  READY1, p=0.08 

NEURO-NAB  DIGIT  Forward 

Lower  scores  for  those  'NOT  READY'  (p=0.075) 

ISAW  GRID  #  correct  under  dual  task 

Lower  scores  for  those  'NOT  READY'(p =0.045) 

Radio  Chatter#  Correct  under  dual  task 

Lower  scores  for  those  'NOT  READY'  (p=0.029) 

Referto  Outliertable 

Conclusions:  Memory  may  be  a  symptom  that  is  readily  self-identified  in  soldiers  with  mTBI,  leading 
them  to  decide  they  are  not  ready  to  undertake  a  deployment  to  a  combat  zone. 

PREDICTIVE  MODELING 

Predictive  modeling  is  in  the  beginning  stages  of  development.  Preliminary  results  are  reported. 

Methods 

Multiple  Logistic  Regression,  Decision  Tree  (CHAID)  and  Discriminant  Analysis  were  used  to 
explore  classification  of  HC  and  mTBI. 

CHAID  offers  a  method  of  efficiently  searching  for  relationships  between  the  independent 
variables  and  the  categorical,  dependent  outcome  measure.  The  output  is  a  tree  diagram  that  shows  the 
category  combinations  that  make  the  most  significant  difference  in  the  outcome  percentages.  Thus, 
CHAID  is  an  effective  tool  for  identifying  segments  that  maximize  the  prediction  accuracy.  In  contrast  to 
logistic  regression  where  participant  with  missing  data  are  dropped,  CHAID  keeps  all  subjects,  using 
whatever  data  is  available.  SPSS  V22.0  was  used  for  this  analysis. 

For  all  methods,  metrics  that  distinguished  groups  from  each  of  the  AMMP  tasks  were  included. 
In  addition,  sensor  data  from  the  ISAW-grid  coordinates  task  for  peak  velocity  was  included. 
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Results 


Logistic  Regression 

Metrics  from  the  CQ  Duty,  ISAW-Grid  and  Patrol-Exertion  tasks  are  included  in  this  optimized 
model  (Table  V.C.l).  This  model  successfully  predicted  80.8%  of  73  participants  included  in  the  analysis, 
88.4%  of  43  HC  and  70%  of  30  mTBI.  This  low  number  of  included  participants  was  due  to  not  having 
completed  a  task  and/or  missing  data.  Additionally,  the  Patrol-Exertion  task  did  not  include  the  first  19 
participants  with  mTBI  because  the  task  was  modified  multiple  times  and  only  those  subjects  who 
completed  the  final  version  of  the  task  were  included. 


Table  VC(1).  Logistic  Regression  Model 


B 

SE 

df 

Sig 

Exp(B) 

95%CI 

Lower 

95%CI 

Upper 

CQ  Duty  #  Visits 

.166 

.072 

1 

.021 

1.180 

1.026 

1.358 

ISAW-Grid;  Coordinates 
Remembered  Dual  task 

-.298 

.169 

1 

.077 

.742 

.533 

1.033 

Patrol  Reaction  Time  End 

.004 

.002 

1 

.106 

1.004 

.999 

1.009 

Patrol  Vision  End 

.938 

.374 

1 

.012 

2.554 

1.227 

5.319 

Constant 

-3.600 

1.934 

1 

.063 

.027 

CHAID 

Task  metrics  from  the  ISAW-Grid  (number  of  grid  coordinates  recalled  under  dual  task 
condition)  and  Patrol-Exertion  (vision  end  of  task  and  reaction  time  end  of  task)  were  included  in  the 
CHAID  model  are  shown  in  Figure  8  where  mTBI  is  coded  as  ‘  V  and  HC  as  ‘O’.  Also  included  is 
readiness  to  deploy  in  72  hours  as  a  yes/no  response.  This  model  successfully  predicted  85.2%  of  108 
participants  included  in  the  analysis,  85.2%  of  54  HC  and  85.2%  of  54  mTBI,  considerably  better  than 
results  from  the  Logistic  Regression  model. 

The  vision  at  end  of  task  (Patrol-Exertion)  split  is  unusual  because  of  the  break  in  sequence  where 
a  value  of  2  is  grouped  with  5,  6,  and  8.  The  median  value  for  the  mTBI  group  is  1.  The  frequency 
distribution  of  the  vision  values  demonstrated  that  ‘2’  was  more  likely  to  be  associated  with  mTBI  and 
therefore  was  included  in  the  right  branch.  The  other  high  values  of  5,  6,  and  8  were  more  likely  to  predict 
mTBI  as  compared  to  HC  so  they  were  also  included  on  the  right  branch.  Missing  values  for  the  mTBI 
group,  where  only  33  of  54  were  available,  may  have  skewed  the  results. 
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Figure  9.  Logistic  Regression  Model 
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Discriminant  Analysis 

The  approach  included  all  variables  entered  in  the  previous  two  models,  and  also  CQ  total  time, 
CQ  rule  breaks,  and  IS  AW  time  under  dual  task  condition  as  well  as  the  total  score  for  the  PCL-C  (PTSD 
screen).  All  variables  were  kept  in  the  model.  Readiness  to  deploy  was  not  included  because  this  method 
requires  continuous  variables.  The  additional  CQ  metrics  were  included  to  evaluate  their  contribution  to 
the  results  given  the  nature  of  the  task  with  its  focus  on  cognitive  function.  Inclusion  of  the  total  PCL-C 
score  for  PTSD  was  hypothesized  to  provide  a  measure  of  classification.  However,  a  final  model  that 
would  be  most  useful  would  probably  not  include  this  metric  since  those  soldiers  with  high  scores  would 
already  be  designated  as  not  ready  to  return  to  duty.  A  total  of  60  participants  were  included  in  this 
analysis  due  to  missing  data,  36  HC  and  24  mTBI. 

In  this  model,  the  PCL-C  sum  score  contributed  the  most  to  classification.  Table  V.C.2  provides  the 
coefficients  associated  with  the  model.  Two  approaches  to  classification  used  provided  the  following 
results: 

•  Modeling  with  all  participants  and  then  classifying:  33  of  36  HC  (91.7%)  and  20  of  24  mTBI 
(83.3%)  were  classified  correctly  for  a  total  88.3%  correctly  classified. 

•  Successive  modeling,  leaving  each  participant  out  of  the  model  and  then  classifying  them  using 
the  resulting  model:  29  of  36  HC  (80.6%)  and  13  of  24  mTBI  (54.2%)  were  classified  correctly 
for  a  total  of  70%  correctly  classified. 

The  second  approach,  an  unbiased  method  of  classification,  yielded  poor  results;  demonstrating  the 
instability  of  the  full  model. 


Table  VC(2).  Linear  Discriminant  Analysis  Function 


Variable 

HC 

mTBI 

Constant 

-527.03 

-541.04 

CQ  Task  Performance 

8.23 

8.45 

CQ  Total  Rule  Breaks 

-0.64 

0.30 

CQ  #  Rules  Broken 

2.25 

0.30 

CQ  Total  Visits 

4.16 

4.45 

CQ  Total  Time(mins) 

0.52 

0.65 

IS  AW  Time  Dual  Task  Condition 

0.91 

0.62 

IS  AW  #  Grid  Coordinates 
Remembered  Under  Dual  Task 
Condition 

1.07 

0.62 

IS  AW  Peak  Velocity 

0.28 

0.27 

Patrol  Vision  End  of  Task 

-0.46 

0.10 

Radio  Chatter  Correct  Under  Dual 
Task  Condition 

66.93 

67.35 

Radio  Chatter  Errors  Under  Dual 
Task  Condition 

42.16 

42.51 

Patrol  Reaction  Beginning  of  Task 

0.04 

0.04 

Patrol  Reaction  End  of  Task 

-0.04 

-0.03 

PCL-C  Sum  (PTSD  Screen) 

0.77 

0.84 

Conclusions 

The  results  of  exploratory  modeling  for  classifying  HC  and  mTBI  were  encouraging.  A  strategic 
plan  for  selection  of  tasks  and  on-going  data  collection  accompanied  with  exploration  of  additional 
metrics  can  yield  models  that  may  be  used  for  determining  readiness  for  return  to  duty. 
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FUTURE  RESEARCH  RECOMMENDATIONS  AND  NEXT  LINES  OF  INQUIRY 


Next  steps  in  research  include  both  general  “gap  area”  themes  as  well  as  specific  research 
question  to  promote  development  and  validation  of  individual  AMMP  tasks.  Both  are  presented 
in  this  section  however  next  steps  in  research  will  admittedly  be  informed  by  DoD/  stakeholder 
needs,  availability  of  test  subjects  within  FORSCOM  or  in  medical  readiness  platforms  in  the 
MEDCOM.  Research  prioritization  may  also  be  a  function  of  specific  interests  and  expertise 
among  clinical  researchers  in  study  design  and  administration  related  to  the  AMMP  tasks. 

AMMP  Battery 

We  recommend  creation  of  an  AMMP  Steering  Committee  during  the  bridge  period  following 
this  current  initial  validation  effort  and  next  steps,  to  assist  with  monitoring  and  potentially 
coordinating  further  validation  studies.  Some  focus  areas  include: 

•  Gap  1:  Tailoring  AMMP  Tasks  to  address  different  types  of  RTD  decision  making 
(Acute  vs.  Chronic  Injury);  Time  since  injury:  Short  (days)  vs.  Long  (months).  The 

current  AMMP  battery  includes  performance  tests  that  could  be  used  to  screen  SM  in 
austere,  far  forward  settings  (to  inform  early  RTD  decision  making  after  acute  injury)  and 
other  AMMP  tasks  likely  better  suited  to  measure  performance  in  established  practice 
environments  (Hospital  Settings).  Follow  on  research  should  be  pursued  to  identify 
optimal  administration  points  in  the  continuum  of  care  and  to  validate  their  use  in 
appropriate  practice  environments.  Considerations  may  include  practical  constraints  to 
administration  (space  or  maximal  time  requirements  to  set  up  and  administer)  or 
equipment  factors  (durability  of  instrumented  systems  (e.g.,  computers  or  accelerometers) 
to  reliably  function  in  austere  environments). 

•  Gap  2:  Standardized  Scoring:  In  current  form,  each  individual  AMMP  task  is  assessed 
based  on  participant  performance  on  a  series  of  task  specific  component  metrics  (e.g., 
time  to  perform,  number  of  errors,  number  of  items  correctly  identified,  etc.).  Thus, 
overall  task  performance  is  based  on  calculating  performance  on  a  series  of  sub¬ 
component  metrics  which  have  not  been  normalized  to  a  standardized  scoring  system 
within  the  battery.  Investigators  see  significant  value  in  the  development  of  a  scoring 
system  based  on  percentages  or  on  a  Likert  based  system  that  specifies:  a)  better  than 
average,  b)  average,  or  c)  worse  than  average,  will  contribute  to  the  clinical  utility  of 
these  assessment  tasks.  This  approach  would  allow  relative  distinctions  between 
performances  on  individual  AMMP  tasks  and  provide  more  specific  feedback  on  a 
participant’s  relative  strengths  and  challenges. 

•  Gap  3:  Phase  II  Discriminant  Validity:  To  meet  with  the  primary  RTD  gap  area  of 
characterizing  readiness  to  RTD,  the  research  team  would  propose  testing  in  a  cohort 
consisting  entirely  of  SM  with  history  of  concussion  to  discriminate  between  those  who 
are  duty  ready  (despite  recent  history  of  concussion)  from  non-duty  ready  personnel  who 
may  appear  functional  but  are  for  some  reason,  non-duty  ready  (e.g.,:  still  symptomatic, 
subtly  impaired  in  one  or  more  physiologic  systems;  or  unable  to  perform  Warrior  Tasks 
or  Battle  Drills  to  standard). 
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•  Responsiveness  to  Change:  If  individual  AMMP  tasks  are  to  be  used  to  measure  progress 
toward  RTD  readiness  as  a  result  of  rehabilitation  (intervention),  research  to  evaluate 
metric  responsiveness  to  change  is  essential.  This  will  dictate  the  need  for  alternate 
forms  of  several  of  the  AMMP  tasks. 

•  Predictive  modeling  using  AMMP  task  metrics  should  focus  on  predicting  ‘Successful 
Return  to  Duty’.  Success  for  different  RTD  roles  must  be  defined  and  studies  that  follow 
the  Service  Member’s  RTD  experience  must  occur  to  generate  the  ultimate  objective  of 
the  AMMP,  to  develop  a  predictive  model  for  successful  RTD.  Factors  which  may  be 
evaluated  might  include  retention  and  promotion,  annual  evaluations,  and  use  of  health 
care  resources  for  continuing  mTBI  symptoms. 

AMMP  SPECIFIC  TASK  DEVELOPMENT 

•  Two  of  the  AMMP  dual-tasks  (ISAW-Grid  and  Load  Magazine-Radio  Chatter)  may  be 
considered  for  use  in  their  current  state  on  a  limited  basis  in  military  rehabilitation  settings  to 
provide  further  information  from  military  clinicians  on  their  utility  and  relationship  to  RTD 
success/failure.  Data  from  this  clinical  use  should  be  evaluated  to  provide  further  validation 
of  these  dual-task  metrics  to  provide  useful  clinical  information  on  patient  performance  and 
progress. 

•  Develop,  test,  and  validate  alternate  version(s)  of  the  CQ  Duty  task  to  allow  testing  for 
responsiveness  to  change  and  response  to  treatment  interventions  focused  on  mTBI 
vulnerabilities  including  executive  dysfunction  and  impaired  attention  and  memory. 

•  Using  the  ISAW-Grid  and  a  modification  of  the  Illinois  Agility-Word  List  task:  Obtain  serial 
measures  of  dual  task  cognitive  load-dynamic  stability  performance  in  a  cohort  of  SMs 
(Active  Duty  Cadets  or  acutely  concussed  Special  Forces  SM)  pre-  and  post-concussion  from 
boxing  or  other  sports  injury  to  establish  recovery  curves  of  static  postural  stability,  dynamic 
stability,  agility,  attention  and  working  memory  in  highly  functional  and  highly  motivated 
personnel  (USMA) 

o  ISAW-Grid  task  at  baseline,  24  hours,  72  hours,  96  hours,  1  week,  9  and  14  days 
to  follow  recovery  from  acute  concussion  and  once  normal  recovery  curves  are 
determined,  to  evaluate  responsiveness  to  interventions  such  as  the  graded  return 
to  activity  protocol. 

o  Illinois  Agility-cognitive  task  (like  Grid-coordinates)  at  baseline,  assuming 

asymptomatic  (headache  and  dizziness  resolved)  and  again  at  5,  7,  9  and  14  days) 
o  Validate  similar  dual-task  performance  study  in  civilian  athlete  cohorts  (e.g., 
academia,  NCAA,  NFL). 

•  Specific  AMMP  tasks  may  have  greater  sensitivity  in  certain  patient  groups  (e.g.  ISAW  Grid 
in  acutely  concussed  personnel,  rolling  component  of  Run  Roll  Aim  for  those  reporting  post 
traumatic  dizziness  as  a  clinical  complaint).  Studies  to  identify  AMMP  task  use  based  on 
patient  complaint  profile  are  needed. 

•  Using  an  up  tempo  video  (this  may  be  a  custom  developed  or  a  ready-made  video  that 
provides  a  busier  and  more  challenging  cognitive  load),  evaluate  the  Patrol-Exertion  task  at 
point  of  RTD  in  established  concussion  care  programs  following  rehabilitation  (given 
exertion  component  and  high  face  validity)  (Candidate  sites:  WOMACK,  NC;  Fort 
Campbell  KY).  Additionally,  develop,  test,  and  validate  alternate  version  Patrol-exertion 
videos  to  include  more  complicated  scenarios  requiring  greater  cognitive  focus  to  allow  use 
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of  the  Patrol-exertion  task  in  testing  for  responsiveness  to  change  and  response  to  treatment 
interventions. 

•  Use  Load  Magazine-  Radio  Chatter  test  to  measure  performance  in  AD  Soldiers  under 
stressed  temperature  conditions  (extreme  hot/  extreme  cold)-  Cross  validate  other  military 
functional  tasks  requiring  bi-manual  dexterity  (USARIEM) 

•  Based  on  findings  from  the  current  inertial  sensor  data  of  the  Illinois  Agility-Packing  List 
and  Run  Roll  Aim  tasks,  consider  further  studies  with  inertial  sensor  instrumentation  to 
evaluate  the  high  level  mobility  and  subtle  mobility  deficits  of  Warrior  athletes  potentially 
with  a  combination  of  motor  tasks  (floor  to  stand  transitions  and  combat  rolls)  that  .challenge 
vestibular  vulnerabilities. 

•  Encourage  additional  studies  to  translate  the  use  of  Dual-task  and  Multitask  assessments  with 
face  validity  to  civilian  tests. 

Conclusions 

The  successful  completion  and  key  finding  of  this  study  provides  ample  reason  to  support  on¬ 
going  research  to  refine  and  apply  the  AMMP  battery  for  evaluating  the  SM  with  mTBI  readiness  for 
successful  return  to  duty.  We  believe  that  tasks  may  be  used  as  well  in  the  civilian  setting  with 
minimal  modification. 


KEY  ACCOMPLISHMENTS  AND  FINDINGS 

1.  Achieved  clinically  acceptable  inter-rater  reliability;  all  test  metrics  for  each  of  the  six  tasks 
remaining  in  the  AMMP  battery  have  interrater  reliability  above  0.90  for  dual-tasks  and 
above  0.88  for  multitasks  except  for  several  metrics  on  errors  and  cues  which  were  then  not 
used  to  evaluate  between  group  differences. 

2.  Based  on  a  minimum  correlation  of  0.3  for  evaluation  of  hypothesized  relationships  between 
standard  neurocognitive  measures  and  AMMP  task  metrics;  these  hypotheses  were 
supported.  Although  small  (almost  all  below  0.4),  expected  correlations  were  demonstrated 
for  most  tasks  with  the  exception  of  the  Patrol-exertion  task. 

a.  Small  correlations  may  be  due  to  complexity  of  the  tasks  with  multiple  factors 
affecting  performance.  In  addition,  the  mTBI  group  was  not  tested  at  the  time  of 
evaluating  AMMP  tasks  for  several  assessments  including  Neurobehavioral 
Assessment  Battery  and  Comprehensive  Trail  Making  Test  tests,  which  may  have 
been  a  confounder  given  lag  times  between  neurocognitive  testing  (obtained  from 
Womack  TBI  Clinic  chart)  and  AMMP  battery  testing. 

b.  Dynamic  Visual  Acuity  Testing  (http://www.micromedical.com/Products/)  was 
administered  as  a  behavioral  measure  of  gaze  stability  and  a  proxy  for  vestibular 
function  testing.  The  DVAT  identified  only  3  subjects  with  mTBI  who  demonstrated 
clinically  significant  impairments  in  gaze  stability  (>  2  lines  lost)  during  rapid  head 
impulses  (>  120  degrees/sec)  relative  to  static  visual  acuity  during  a  head  stationary 
condition.  This  clinical  reference  point  was  consequently  insufficient  for  use  as  a 
correlational  measure  for  AMMP  metrics  intended  to  target  vestibular  vulnerabilities. 

3.  Subjects  with  mTBI  appear  to  have  more  difficulty  than  HC  with  cognitive  tasks  especially 
when  overlaid  on  physical  challenge: 

a.  Dual-task  interference  for  the  cognitive  task  when  tested  during  the  motor  task  was  able  to 
distinguish  HC  from  mTBI  groups  in  the  Load  Magazine-Radio  Chatter  and  Instrumented 
Stand  and  Walk-Grid  Coordinates  Dual  Task. 
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b.  Metrics  focusing  on  executive  function  (cognitive)  vulnerabilities  demonstrated  the  ability  to 
distinguish  groups  in  the  Charge  of  Quarters  Duty  task.  A  combination  of  exertional, 
cognitive  and  reaction  time  requirements  in  the  Patrol-exertion  task  resulted  in 
symptom  report  of  reduced  visual  clarity  and  slower  reaction  time  metrics  which 
distinguished  mTBI  compared  to  HC. 

4.  Other  than  reduced  peak  velocities  during  180  degree  turns  during  normal  walking  as 
measured  by  inertial  sensors,  and  reaction  times  during  the  mulitask,  Patrol-exertion,  AMMP 
physical  performance  metrics  alone  were  insufficient  to  distinguish  groups,  leading  to 
speculation  that  for  the  most  part  subtle  differences  may  require  inertial  sensors  or  other 
laboratory  equipment  to  identify  physical  performance  dissimilarities  in  elite  Warfighters 
that  are  healthy  versus  concussed. 

5.  Using  1)  the  AMMP  task  metrics  that  discriminate  groups,  2)  PCL-C  scores  (PTSD  screen), 
and  3)  self-assessment  for  readiness  to  return  to  combat  duty,  predictive  modeling  through 
logistic  regression,  discriminant  analysis  and  decision  tree  analysis  (CHAID)  yielded 
successful  classifications  of  HC  and  mTBI  between  80.8%  and  91.7%  of  cases.  Accepting  a 
number  of  limitations  such  as  an  inability  to  use  cases  with  missing  data,  these  methods  show 
promise  in  identifying  specific  factors  and  AMMP  metrics  that  provide  the  most  information 
and  that  may  be  further  investigated  for  RTD  decision-making. 

6.  Exploratory  analyses  using  time  series  analysis  methodologies  (principal  component  analysis 
(PC A))  is  demonstrating  the  potential  to  correctly  categorize  healthy  control  and  subjects 
with  mTBI.  These  exploratory  findings  require  further  investigation  with  the  current  data  set 
to  develop  a  theoretical  framework  for  their  interpretation  and  association  with  known  mTBI 
related  vulnerabilities.  . 

7.  There  is  likely  an  optimal  balance  of  symptom  report,  impairment  and  functional  metrics  that 
can  be  modeled  to  optimally  discriminate  between  ready  and  non-duty  ready  personnel. 

8.  An  AMMP  Training  Manual  and  Training  Modules  (PowerPoint  and  Video)  are  included  in 
appendices  and  under  separate  files.  For  each  of  the  six  AMMP  tasks,  these  materials  provide  1) 
Task  Description  and  Set  Up,  2)  Examiner  Instructions  and  Script,  3)  Score  Sheet,  4)  Scoring  Guide, 
5)  Task  specific  testing  materials.  The  AMMP  Training  Manual  and  Modules  are  designed  to  enable 
prospective  AMMP  examiners  to  understand,  visualize,  and  practice  the  administration  of  the  6 
AMMP  tasks. 
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REPORTABLE  OUTCOMES 


LIST  OF  MANUSCRIPTS,  ABSTRACTS,  PRESENTATIONS 

Manuscripts  (published) 

Scherer  M,  Weightman  M,  Radomski  M,  Davidson  L,  McCulloch  K.  “Returning  Service  Members  to 
Duty  Following  Mild  Traumatic  Brain  Injury:  Exploring  the  Use  of  Dual-Task  and  Multitask  Assessment 
Methods”,  Phys  Ther.  2013;  93:1255-1267. 

Smith,  LB,  Radomski,  MV,  Davidson  LF,  Finkelstein,  M,  Weightman,  MM,  Scherer,  MR,  McCulloch,  K 
(2014).  Development  and  preliminary  reliability  of  a  multitasking  assessment  following  concussion. 
American  Journal  of  Occupational  Therapy,  68,  439-443. 

Manuscripts  (submitted  or  in  preparation) 

Weightman  MM,  McCulloch  KL,  Radomski  MV,  Finkelstein  M,  Cecchini  A,  Davidson  L,  Heaton  KJ, 
Smith  L,  Scherer  M.  Further  development  of  the  Assessment  of  Military  Multitasking  Performance: 
Iterative  Reliability  Testing.  Submitted  to  the  Journal  of  Rehabilitation  Research  &  Development  (20 
October  2015). 

McCulloch,  K,  Cecchini  A,  McMillan  H,  Cleveland  C,  Davidson  LF,  Finkelstein  M,  Radomski  M, 
Scherer  M,  Smith  L,  Weightman  M,  Military/Civilian  Collaborations  for  mTBI  Rehabilitation  Research 
in  an  Active  Duty  Population:  Lessons  Learned  from  the  Assessment  of  Military  Multitask  Performance 
Project,  (2015  in  preparation). 


Presentations 


Date  of  Presentation 

Title,  Name  of  Conference,  and  Location 

November  6,  2013 
(platform) 

Returning  Service  Members  to  duty  following  mild  traumatic  brain 
injury:  Exploring  the  use  of  dual-  and  multitask  assessment  methods 
(McCulloch  &  Scherer),  Association  of  Military  Surgeons  of  the 
United  States,  Seattle,  WA 

February  21,  2014 
(poster) 

Assessing  Motor  Performance  in  Military  Service  Members 

Following  mTBI  Using  Inertial  Sensor  Data  (McCulloch  &  Yu), 
Human  Movement  Science  Student  Research  Day,  Chapel  Hill,  NC 

March  19-22,  2014 
(poster) 

Preliminary  inter-rater  reliability  for  a  novel  dual-task  &  multitask 
assessment  battery  guiding  return-to-duty  in  concussed  Service 
Members  (Weightman,  McCulloch,  Davidson,  Scherer,  Smith, 
Finkelstein,  Radomski),  International  Brain  Injury  Association,  San 
Francisco,  CA 

April  11,  2014 
(workshop) 

Development  of  a  Dual-and  Multitask  Assessment  to  Inform  Return 
to  Duty  after  Concussion:  Implications  for  Civilian  Practice 
(Weightman,  Radomski),  Minnesota  Brain  Injury  Association, 
Minneapolis,  MN 

June  11-14,  2014 
(symposium) 

State  of  the  Art  Military  TBI  Care:  Latest  Policies  and  Innovations 
in  Return  to  Duty  Assessment  (Scherer,  McCulloch,  Maxfield- 
Panker,  MacMillan),  American  Physical  Therapy  Association 
(APTA)  2014  Annual  conference,  Charlotte,  NC 
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August  18-21,  2014 
(paper) 

A  novel  dual-task  and  multitask  assessment  battery  guiding  return- 
to-duty  in  concussed  Service  Members  (Smith, 

Scherer,  Weightman),  3rd  International  Congress  of  Soldiers’ 

Physical  Performance,  Boston,  MA 

October  8  -  12,  2014 
(posters)* 

Development  of  the  Run-Roll-Aim  Task,  a  high  level  mobility  task 
assessing  physical  skills  related  to  military  duty  and 

Development  of  the  dual-task  Illinois  Agility  task,  a  component  of  the 
Assessment  of  Military  Multitasking  Performance,  McCulloch  & 
Goldberg,  American  Congress  of  Rehabilitation  Medicine  (ACRM) 
Annual  Meeting,  Toronto,  ON 

February  5,  2015 
(platform  presentation) 

Inertial  sensors  detect  subtle  mobility  differences  in  soldiers  with 
persistent  concussion  symptoms:  preliminary  findings  for  the 
instrumented  stand  and  walk  (Scherer,  Finkelstein,  McCulloch, 

Smith,  Weightman),  APTA  Combined  Sections  Meeting, 

Indianapolis,  IN 

May  5,  2015 
(poster) 

Inter-rater  reliability  of  the  Assessment  of  Military  Multitasking 
Performance:  A  test  battery  for  concussed  Service  Members, 

Cecchini,  McMillan,  Cleveland,  Weightman,  McCulloch,  Womack 
Army  Medical  Center  Research  Symposium,  Fort  Bragg,  NC 

September  2,  2015 
(Report  to  DoD  Return 
to  Duty  Symposium) 

Assessment  of  Military  Multitasking  Performance  (Weightman, 
Radomski,  Scherer),  Return  to  Duty  Toolkit  Working  Group 
Symposium,  Fort  Detrick,  MD 

October  29,  2015 
(symposia) 

Assessment  of  Military  Multitasking  Performance  multi-task 
components:  Informing  return  to  duty  after  concussion  and 

Assessment  of  Military  Multitasking  Performance  dual-task 
components:  Informing  return  to  duty  after  concussion 
(Weightman,  Radomski,  Scherer,  McCulloch),  ACRM  Annual 
Meeting,  Dallas,  TX 

October  28-30,  2015 
(poster)* 

High  level  mobility  task  analysis  after  military  mild  traumatic  brain 
injury  identifies  subtle  motor  control  impairments  (McCulloch, 
Favarov,  Balcilar),  ACRM  Annual  Conference,  Dallas,  TX 

October  28-30,  2015 
(poster)* 

Instrumented  stand  and  walk  in  single  and  dual-task  conditions  in 
Soldiers  with  persistent  concussion  symptoms ,  (Weightman, 
Finkelstein,  McCulloch,  Scherer),  ACRM  Annual  Conference, 

Dallas,  TX 

Accepted  for  February 
2016 

(symposium) 

Validation  of  the  Assessment  of  Military  Multitasking  Performance 
for  Mild  TBI.  Federal  Section,  American  Physical  Therapy 
Association’s  Combined  Section  Meeting,  (Weightman,  Scherer, 
McCulloch),  Anaheim,  CA 

Accepted  for  July  2016 
(poster) 

Non-linear  analyses  of  Agility  Task  Performance  after  military  mild 
traumatic  brain  injury :  An  approach  to  identify  subtle  motor  control 
impairments  (McCulloch,  Favorov,  Weightman),  IVSTEP, 

Columbus,  OH 
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Published  abstracts 

*These  poster  presentation  abstracts  will  be  published  in  a  future  Supplement  to  the  Archives  of  Physical 
Medicine  and  Rehabilitation. 


FUNDING  APPLIED  FOR  BASED  ON  WORK  SUPPORTED  BY  THIS  AWARD 

Study  title:  Return  to  Duty  after  Concussion  with  Exertional  Symptoms:  Physiologic  and 
Functional  Predictor 

PI:  Karen  McCulloch,  PT,  PhD,  NCS  for  proposal;  Ft  Bragg  local  PI  CAPT  Henry  McMillan,  PT,  DPT, 
MBA 

Funder  submitted  to  and  date  of  submission:  CDMRP  -  Neurosensory  call  -  invited  to  write  full  proposal 
following  pre -proposal  review;  was  recommended  for  funding  as  an  alternate,  was  not  funded.  Will  be 
rewriting  this  project  with  slightly  modified  research  plan/aims  for  submission  under  the  BAA.  Full 
proposal  submitted  February  2015. 

Specific  Aims 

•  Describe  patterns  of  symptom  complaint,  impairments  and  recovery  for  SM  with  persistent  physical 

symptoms  after  mTBI. 

•  Confirm  feasibility  and  safety  of  a  graded  return  to  activity  program,  describing  efficacy  and 

intervention  duration  for  SMs  with  persistent  exertional  symptoms. 

•  Examine  association  of  physiologic,  functional  and  simulation-based  measures  that  reflect  RTD 

readiness;  identify  predictors  of  successful  RTD  for  injured  SMs. 

Relevance 

Given  the  focus  of  this  proposed  study  was  on  exertional  complaints  following  mTBI,  we  intended  to  use 
exertional  tasks  from  the  AMMP  as  a  component  of  the  study  (including  Run-Roll-Aim,  Illinois  Agility 
Test,  Patrol-Exertion  task).  We  also  intended  to  be  able  to  address  responsiveness  of  these  AMMP  tasks 
given  the  longitudinal  nature  of  the  study,  as  well  as  validate  AMMP  findings  to  the  clinical  judgment 
about  readiness  to  return  to  duty  based  on  the  current  practice  standards  at  Womack  Army  Medical 
Center. 

Study  title:  Development,  Reliability,  and  Equivalence  of  an  Alternative  Form  for  the  CQ  Duty 

Performance-based  Measure 

PI:  Mary  Vining  Radomski,  PhD,  OTR/L 

Funder  submitted  to  and  date  of  submission:  USAMRMC  BAA,  Full  Proposal  August  2015 
Specific  Aims 

•  Develop  an  alternate  form  of  the  CQ  Duty. 

•  Assure  rater  agreement  across  2  raters. 

•  Evaluate  equivalence  of  CQ  Duty-Alternate  Form. 

Relevance 

Performance-based  assessments  involving  multitasking  cannot  be  repeated  because  of  significant  learning 
effects.  If  we  succeed  in  developing  a  CQ  Duty- Alternate  Form,  the  CQ  Duty  may  be  used  to  both 
identify  executive  dysfunction  and  quantify  treatment  outcomes  in  SM  with  mTBI.  In  addition,  we  may 
be  able  to  demonstrate  a  potentially  replicable  process  that  could  be  used  to  develop  alternate  forms  for 
other  performance-based  measures  of  high-level  cognitive  skills  and  abilities. 

Study  title:  The  Front-Desk  Duty  Multitasking  Test  for  Adults  with  Mild  Stroke:  A  Pilot  study 

PI:  Mary  Vining  Radomski,  PhD,  OTR/L 

Funder  submitted  to  and  date  of  submission:  Courage  Kenny  Foundation,  October  2015 
Specific  Aims 

•  Optimize  all  Front  Desk  Duty  (FDD)  materials  received  from  Shenandoah  University. 
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•  Train  6  FDD  research  team  occupational  therapists  (two  from  each  of  CKRI  at  Abbott 
Northwestern,  Mercy,  and  United  Hospitals)  and  evaluate  competence  in  administering  and 
scoring  the  FDD. 

•  Evaluate  IRR  of  the  FDD  on  21  study  participants. 

•  Determine  between-groups  discrimination  of  the  FDD  when  administered  to  34  individuals  with 
mild  stroke  recruited  from  CKRI  at  Abbott  Northwestern,  Mercy,  and  United  hospitals  and  34 
healthy  individuals. 

•  Examine  the  relationship  between  FDD  scores  and  that  of  neurocognitive  measures  of  EF  and 
self-reported  EF  in  daily  life. 

Relevance 

The  Front  Desk  Duty  assessment  is  a  civilianized  version  of  the  CQ  Duty,  with  the  same  task  and  scoring 
but  within  the  context  of  a  shift  at  the  front  desk  of  a  motel.  If  funded,  this  study  may  capitalize  on 
findings  from  the  AMMP  study  to  advance  rehabilitation  practice. 
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CONCLUSION 


The  Assessment  of  Military  Multitasking  Performance  was  developed  as  functional  return  to  duty 
assessment  battery  in  response  to  a  critical  gap  in  the  DoD’s  mTBI  management  policy.  The  AMMP  was 
developed  in  response  to  stakeholder  requirements  for  face  validity,  clinical  feasibility  and  the  versatility 
to  address  a  broad  spectrum  of  known  mTBI  related  vulnerabilities.  Objective  findings  from  this  research 
project  demonstrate  that  the  AMMP  is  feasible  with  regard  to  equipment,  setup  time  and  space,  time  to 
administer,  and  scoring.  Findings  demonstrate  that  clinically  acceptable  interrater  reliability  of  scoring 
was  clearly  demonstrated  in  all  AMMP  tasks.  In  the  domain  of  discriminant  validity,  four  of  six  tasks 
included  component  metrics  that  distinguished  HC  from  mTBI  groups.  When  instrumentation  data  are 
considered  in  addition  to  clinical  outcomes,  accelerometer  data  recorded  in  three  tasks  provided  enhanced 
resolution  on  subtle  kinematic  performance  differences  between  groups  that  may  prove  useful  for 
informing  duty  readiness  in  more  chronic,  or  less  obvious  RTD  determinations.  In  general,  more 
demanding,  dynamic  conditions  were  necessary  to  distinguish  between  HC  and  mTBI  groups  suggesting 
that  the  difficulty  level  of  physical  tests  will  continue  to  be  a  key  consideration  for  testing  in  the 
premorbidly  high  functioning  Warrior  population. 

Given  the  heterogeneous  presentation  of  mTBI,  and  the  imperative  for  “making  the  right  call”  to  return  a 
Warrior  to  full  Duty  (i.e.,  combat,  during  time  of  war)  -  there  is  a  clinical  research  imperative  to  further 
explore  and  validate  predictive  modeling  techniques  to  accurately  discriminate  between  duty  ready  and 
non-duty  ready  personnel.  Future  validation  efforts  using  AMMP  tasks  as  regressors  in  such  a  model 
should  be  prioritized  in  conjunction  with  other  clinical  tests,  symptom  report  scales,  or  other  viable 
markers  to  account  for  the  variance  associated  with  duty  readiness. 
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APPENDIX  1:  LIST  OF  PERSONNEL  RECEIVING  PAY  FROM  THE  RESEARCH  EFFORT 


Project  Participants  and  Collaborating  Organizations — Paid  by  Award 

Name 

Institution 

Project  Role 

Principal  Contributions 

Margaret  M. 

(Maggie) 

Weightman  PT,  PhD 

Courage  Kenny  Research 
Center  (CKRC)/  Allina 
Health 

Principal 

Investigator/  Project 
Director 

Manage,  oversee  and 
contribute  to  all  aspects  of 
project.  AMMPtask 
development. 

Mary  Vining 
Radomski 

CKRC 

Co-Principal 

Investigator 

Focus  on  development  of 
training  manual  and 
videos;  assist  with  project 
oversight.  AMMP  task 
development. 

Marsha  Finkelstein 
MS 

CKRC 

Co-Investigator 

Oversee  all  aspects  of  data 
entry  and  data  analysis 

Amy  Cecchini,  PT, 
MS 

CKRC  -Contractor 

Research 

Coordinator 

Coordinate  project  and 
oversight  at  Fort  Bragg 
including  recruitment  and 
testing  of  subjects 

Caroline  Cleveland 
BS 
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Research  Assistant 
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and  testing  all  subjects  at 
Fort  Bragg,  REDCap  data 
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Institute  for  Creative 
Technologies,  University 
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CKRC  -Contractor 
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Development  of  videos  for 
Patrol-Exertion  task  using 
Virtual  Iraq- Virtual 
Afghanistan  software 

Daniel  Nilsson 

CKRC  -Contractor 

Engineer 

Engineering  Consultant, 
development  of  reaction 
time  apparatus  for  Patrol- 
exertion  task,  assist  with 
inertial  sensor  analysis 
ISAW-grid  task 

Rachel  Ahn 

CKRC-Contractor 

Graduate  student 

Data  entry  and  verification 

Dane  Bonath 

CKRC  -Contractor 

Graduate  student 

Data  entry  and  verification 

John  Hughes 

University  of  MN, 
Biostatistics 

CKRC  -Contractor 

Consultant 

Consult  and  assist  with 
data  analysis 

Robert  Kreiger 

University  of  MN 

CKRC  -Contractor 

Consultant 

Consult  and  assist  with 
data  analysis,  exploratory 
analyses 

Emily  Shoemaker 

CKRC 

Technical  writer 

Assist  with  final  report 

Anthony  Abeln 

CKRC 

Occupational 

Therapist 

Pilot  testing 

Matt  White 

CKRC 

Occupational 

Therapist 

Pilot  testing 

Kristina  Kath 

CKRC 

Administrative 

Assistant 

Administrative  assistance 
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Department  of 

Biostatistics 

CKRC  -Contractor 
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Leslie  Davidson 
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LLC 

Co-investigator 

Development  of  training 
videos  and  materials, 
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Engineering,  UNC-Chapel 
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Graduate  student- 
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engineering 
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Mary  Beth  Osborne 

Division  of  Physical 
Therapy  UNC-Chapel  Hill 

Assistant 

Administrative  assistance/ 
cover  release  time 

Pro  ject  Participants  and  Collaborating  Organizations — NOT  PAID ,  in  kind  contributions,  and 
DoD  collaborators 

Name 

Institution 

Project  Role: 

Principal  Contributions 

MAJ  Matthew  Scherer, 
PT,  PhD 

Andrew  Rader  US 

Army  Health  Clinic,  JB 
Fort  Myer-  Henderson 
Hall,  VA 

(previously  USARIEM, 
Natick,  MA) 

Co-investigator 

AMMP  task 
development,  IRB 
approval/supervision 
for  USARIEM, 

CPT  Laurel  Smith  MA, 
OTR/L 

United  States  Army 
Research  Institute  of 
Environmental 

Medicine,  Natick  MA 

Co-investigator 

AMMP  task 
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approval/supervision 
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Kristin  Heaton  PhD 

United  States  Army 
Research  Institute  of 
Environmental 
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Neuropsychology 

consultant 
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Injury  Medicine, 
Womack  Army 

Medical  Center,  Fort 
Bragg,  NC 

Fort  Bragg  Principal 
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Bragg  staff  and 
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Muhammet  Balcilar 

Department  of 
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Engineering,  UNC- 
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Graduate  student 

Analysis  of  inertial 
sensor  data  for  Illinois 
Agility  and  Run  Roll 
Aim  tasks 

Lars  Oddsson  PhD 

Courage  Kenny 

Research  Center 
(CKRI)/  Allina  Health 

Consultant 

Advisor  on  inertial 

sensor  and 

biomechanical  analysis 

Various  audiometry 
staff 

US  Army  Aeromedical 
Research  Laboratory, 
Fort  Rucker,  AL 

Consultants 

Consult  and  assist  with 
development  of  radio 
chatter  audio  files. 
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APPENDIX  2 

AMMP  MANUAL  -  Overview 
(See  AMMP  Manual  in  a  separate  file.) 


The  Assessment  of  Military  Multitasking  Performance  (AMMP)  is  a  recently-developed  performance- 
based  test  battery  that  has  been  subjected  to  preliminary  validation  to  evaluate  inter-rater  reliability  and 
known-groups  discrimination.  The  AMMP  was  developed  by  an  interdisciplinary  team  of  civilian  and 
military  rehabilitation  researchers  to  help  inform  duty -readiness  decision-making  for  Service  Members 
(SM)  who  have  sustained  a  concussion/mild  traumatic  brain  injury. 

Introduction  to  the  AMMP  Tasks 

The  six  tasks  that  compose  the  AMMP  were  developed  using  either  a  dual-task  or  multitasking  paradigm. 
All  tasks  employ  observational  metrics  and  some  tasks  also  utilize  inertial  sensor  data  to  characterize  SM 
performance. 

AMMP  Multitasks 
Charge  of  Quarter  Duty 

The  SM  is  challenged  to  develop  and  execute  a  work  plan  for  completing  an  array  of  interleaving  tasks 
(supply  inventory,  PVC  foot  stool  assembly,  providing  information  to  superiors,  prospective  memory 
tasks)  associated  with  his/her  hypothetical  assignment  to  Charge  of  Quarters  Duty. 

Run-Roll-Aim 

The  SM  completes  a  high  level  mobility  task  with  multiple  maneuvers  while  carrying  a  simulated 
weapon.  Maneuvers  are  cued  by  a  computer  screen  with  a  handheld  remote  controlled  slide  advancer. 

The  task  requires  a  rapid  start,  avoiding  a  “trip  wire”  obstacle,  performing  a  3-5  second  rush,  combat 
rolling,  searching  for  visual  targets  through  simulated  weapon  scope,  rapid  lateral  dodging  and  back 
pedaling. 

Patrol-Exertion 

The  SM  is  challenged  to  gather  intelligence  in  a  recorded  video  depicting  a  virtual  Afghanistan  patrol 
environment  while  reporting  observed  IED  markers  based  on  a  briefing  provided  at  the  beginning  of  the 
video.  The  SM  then  uses  the  information  to  answer  specific  questions  from  memory  at  the  end  of  the 
patrol  video.  The  SM  will  perform  continuous  step-ups  on  an  exercise  step  at  an  intensity  of  65-85%  of 
HR  maximum  throughout  the  activity  while  being  monitored  for  effort  level  via  a  Polar  HR  monitor  and 
performance  observation.  The  SM  will  be  wearing  a  combat  helmet,  eye  protection,  and  be  carrying  a 
simulated  M16  weapon  equipped  with  a  trigger  switch  connected  via  Bluetooth  to  a  computer  configured 
to  record  reaction  time  (RT).  The  SM  is  required  to  press  the  switch  each  time  a  beep  tone  stimulus  is 
heard  throughout  the  video  as  a  measure  of  RT  during  a  divided  attention  multitask. 

AMMP  Dual-tasks 

Illinois  Agility  Test  -  Packing  List 

The  Illinois  Agility  Test  requires  running  distances  of  30’  with  rapid  direction  changes  and  navigation  of 
obstacles  in  a  serpentine  pattern  during  the  middle  part  of  the  obstacle  course.  A  memory  task  is  also 
completed.  Then  both  the  agility  task  and  the  memory  task  are  performed  at  the  same  time.  Accuracy  of 
memory  recall  and  time  to  complete  the  agility  task  are  measured  in  single  and  dual-task  conditions. 

Instrumented  Stand  and  Walk  -  Grid  Coordinates 

The  SM  is  challenged  to  perform  the  Instrumented  Stand  and  Walk  (ISAW)  test  (developed  by  APDM) 
which  includes  instrumented  and  timed  assessment  of  quiet  standing  for  30  seconds,  assessment  of 
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dynamic  stability  during  walking  for  two  7  m  (23  foot)  lengths  with  a  180  degree  turn  at  midpoint.  The 
SM  will  next  memorize  an  8  digit  alphanumeric  grid  coordinate  provided  within  the  context  of  a 
simulated  patrol  mission  brief  and  report  the  exact  sequence  back  to  the  examiner  after  45  seconds. 
Finally,  both  the  IS  AW  and  the  grid  memorization  tasks  will  be  performed  simultaneously.  Accuracy  of 
grid  coordinate  recall,  postural  sway  area,  gait  path  variability,  and  time  to  complete  the  ISAW  (i.e.  gait 
speed)  will  be  measured  in  single  and  dual-task  conditions. 

Load  Magazine  -  Radio  Chatter 

SM  completes  a  relatively  automatic  manual  task  choosing  from  a  bin  of  mixed  size  dummy  rounds  (5.56 
and  7.62  caliber)  and  loading  5.56  caliber  training  rounds  into  magazines  as  fast  as  possible  both  in  a 
single  and  a  dual-task  condition.  The  dual-task  condition  requires  monitoring  radio  communication  and 
verbally  announcing  when  radio  chatter  is  relevant  to  scenario  instructions. 


Structure  of  the  AMMP  Manual 

In  this  Manual,  there  is  a  chapter  for  each  of  the  aforementioned  AMMP  tasks.  The  following  sections 
compose  each  test-task  specific  chapter: 

•  Task  Description  and  Set  Up 

•  Examiner  Instructions  and  Script 

•  Score  Sheet 

•  Scoring  Guide 

•  Materials 


Guidance  for  Use  of  the  AMMP  Manual 

Administration  of  the  AMMP  requires  that  examiners  understand  the  theoretical  foundation  of  the  tasks 
and  are  competent  in  conducting  and  scoring  the  tasks. 

•  To  understand  the  theoretical  foundations  of  the  AMMP,  examiners  should  be  familiar  with 
papers  that  have  been  published  on  this  test  (e.g.,  Radomski  et  al.,  2013;  Scherer  et  al.,  2013; 
Smith  et  al.,  2014). 

•  To  assure  competence,  examiners  should  first  read  the  AMMP  Manual  in  its  entirety.  Next, 
examiners  should  watch  the  AMMP  Training  Modules.  After  doing  so,  at  least  three  practice 
administrations  of  each  test  task  are  recommended  before  administering  the  test  task  on  a  subject 
or  patient.  This  will  enable  the  examiner  to  become  comfortable  simultaneously  reading  the 
administration  script,  managing  any  equipment,  and  observing  performance. 
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Returning  service  members  to  duty  following  mild  traumatic  brain  injury:  exploring  the  use 
of  dual-task  and  multitask  assessment  methods.  Physical  Therapy ,  93(9),  1254-1267. 
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(2014).  Development  and  preliminary  reliability  of  a  multitasking  assessment  following  concussion. 
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APPENDIX  3 

AMMP  TRAINING  MODULES  -  Overview 
(See  AMMP  Training  Modules  in  a  separate  file.) 


Description  and  Purpose  of  the  AMMP  Training  Modules 

The  AMMP  Training  Modules  are  designed  to  enable  prospective  AMMP  examiners  to  understand  and 
visualize  the  administration  of  the  6  AMMP  tasks.  Careful  study  of  the  AMMP  Manual  and  independent 
practice  are  the  primary  means  by  which  researchers  and  clinicians  learn  to  administer  the  AMMP;  the 
AMMP  Training  Modules  are  intended  to  complement,  not  replace,  the  AMMP  Manual  and  independent 
practice. 


AMMP  Training  Modules  Learning  Objectives 

After  using  the  AMMP  Training  Modules  and  carefully  studying  the  AMMP  Manual,  prospective  AMMP 
examiners  will  be: 

•  Able  to  correctly  administer  and  score  all  6  AMMP  test-tasks  as  detailed  in  the  AMMP  Manual; 

•  Familiar  with  common  challenges  associated  with  test-task  set  up,  administration,  and  scoring; 

•  Aware  of  red  flags  to  stop  the  testing  session. 


Structure  of  the  AMMP  Training  Modules 

There  are  6  Training  Modules  (Powerpoint  slide  sets)  that  correspond  to  each  of  the  AMMP  Tasks.  Each 
Module  provides  a  brief  description  of  the  task,  describes  scoring  metrics,  and  includes  a  brief  video  of 
test  task  performance. 


Guidance  for  Use  of  the  AMMP  Training  Modules 

Administration  of  the  AMMP  requires  that  examiners  understand  the  theoretical  foundation  of  the  tasks 
and  are  competent  in  conducting  and  scoring  the  tasks. 

•  To  understand  the  theoretical  foundations  of  the  AMMP,  examiners  should  be  familiar  with 
papers  that  have  been  published  on  this  test  (e.g.,  Radomski  et  al.,  2013;  Scherer  et  al.,  2013; 
Smith  et  al.,  2014). 

•  To  assure  competence,  examiners  should  first  read  the  AMMP  Manual  in  its  entirety.  Next, 
examiners  should  watch  the  AMMP  Training  Modules.  After  doing  so,  at  least  three  practice 
administrations  of  each  test  task  are  recommended  before  administering  the  test  task  on  a  subject 
or  patient.  This  will  enable  the  examiner  to  become  comfortable  simultaneously  reading  the 
administration  script,  managing  any  equipment,  and  observing  performance. 

References 
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INTRODUCTION 


Overview  and  Purpose  of  the  Assessment  of  Military  Multitasking  Performance  (AMMP) 

The  Assessment  of  Military  Multitasking  Performance  (AMMP)  is  a  recently-developed 
performance-based  test  battery  that  has  been  subjected  to  preliminary  validation  to  evaluate 
inter-rater  reliability  and  known-groups  discrimination.  The  AMMP  was  developed  by  an 
interdisciplinary  team  of  civilian  and  military  rehabilitation  researchers  to  help  inform  duty- 
readiness  decision-making  for  Service  Members  (SM)  who  have  sustained  a  concussion/mild 
traumatic  brain  injury. 

Introduction  to  the  AMMP  Test  Tasks 

The  six  test  tasks  that  compose  the  AMMP  were  developed  using  either  a  dual-task  or 
multitasking  paradigm.  All  test  tasks  employ  observational  metrics  and  some  test  tasks  also 
utilize  inertial  sensor  data  to  characterize  SM  performance.  At  present,  there  is  no  prescribed  task 
administration  order. 

AMMP  Multitasks 
Charge  of  Quarter  Duty 

The  SM  is  challenged  to  develop  and  execute  a  work  plan  for  completing  an  array  of  interleaving 
tasks  (supply  inventory,  PVC  foot  stool  assembly,  providing  information  to  superiors, 
prospective  memory  tasks)  associated  with  his/her  hypothetical  assignment  to  Charge  of 
Quarters  Duty. 

Run-Roll-Aim 

The  SM  completes  a  high  level  mobility  task  with  multiple  maneuvers  while  carrying  a 
simulated  weapon.  Maneuvers  are  cued  by  a  computer  screen  with  a  handheld  remote  controlled 
slide  advancer.  The  task  requires  a  rapid  start,  avoiding  a  “trip  wire”  obstacle,  performing  a  3-5 
second  rush,  combat  rolling,  searching  for  visual  targets  through  simulated  weapon  scope,  rapid 
lateral  dodging  and  back  pedaling. 

Patrol-Exertion 

The  SM  is  challenged  to  gather  intelligence  in  a  recorded  video  depicting  a  virtual  Afghanistan 
patrol  environment  while  reporting  observed  IED  markers  based  on  a  briefing  provided  at  the 
beginning  of  the  video.  The  SM  then  uses  the  information  to  answer  specific  questions  from 
memory  at  the  end  of  the  patrol  video.  The  SM  will  perform  continuous  step-ups  on  an  exercise 
step  at  an  intensity  of  65-85%  of  HR  maximum  throughout  the  activity  while  being  monitored 
for  effort  level  via  a  Polar  HR  monitor  and  performance  observation.  The  SM  will  be  wearing  a 
combat  helmet,  eye  protection,  and  be  carrying  a  simulated  M16  weapon  equipped  with  a  trigger 
switch  connected  via  Bluetooth  to  a  computer  configured  to  record  reaction  time  (RT).  The  SM 
is  required  to  press  the  switch  each  time  a  beep  tone  stimulus  is  heard  throughout  the  video  as  a 
measure  of  RT  during  a  divided  attention  multitask. 


AMMP  Dual-tasks 

Illinois  Agility  Test  -  Packing  List 

The  Illinois  Agility  Test  requires  running  distances  of  30’  with  rapid  direction  changes  and 
navigation  of  obstacles  in  a  serpentine  pattern  during  the  middle  part  of  the  obstacle  course.  A 
memory  task  is  also  completed.  Then  both  the  agility  task  and  the  memory  task  are  performed  at 
the  same  time.  Accuracy  of  memory  recall  and  time  to  complete  the  agility  task  are  measured  in 
single  and  dual-task  conditions. 

Instrumented  Stand  and  Walk  -  Grid  Coordinates 

The  SM  is  challenged  to  perform  the  Instrumented  Stand  and  Walk  (IS AW)  test  (developed  by 
APDM)  which  includes  instrumented  and  timed  assessment  of  quiet  standing  for  30  seconds, 
assessment  of  dynamic  stability  during  walking  for  two  7  m  (23  foot)  lengths  with  a  1 80  degree 
turn  at  midpoint.  The  SM  will  next  memorize  an  8  digit  alphanumeric  grid  coordinate  provided 
within  the  context  of  a  simulated  patrol  mission  brief  and  report  the  exact  sequence  back  to  the 
examiner  after  45  seconds.  Finally,  both  the  ISAW  and  the  grid  memorization  tasks  will  be 
performed  simultaneously.  Accuracy  of  grid  coordinate  recall,  postural  sway  area,  gait  path 
variability,  and  time  to  complete  the  ISAW  (i.e.  gait  speed)  will  be  measured  in  single  and  dual¬ 
task  conditions. 

Load  Magazine  -  Radio  Chatter 

SM  completes  a  relatively  automatic  manual  task  choosing  from  a  bin  of  mixed  size  dummy 
rounds  (5.56  and  7.62  caliber)  and  loading  5.56  caliber  training  rounds  into  magazines  as  fast  as 
possible  both  in  a  single  and  a  dual  task  condition.  The  dual-task  condition  requires  monitoring 
radio  communication  and  verbally  announcing  when  radio  chatter  is  relevant  to  scenario 
instructions. 


Structure  of  the  AMMP  Manual 

In  this  Manual,  there  is  a  chapter  for  each  of  the  aforementioned  AMMP  test  tasks.  The 
following  sections  compose  each  test-task  specific  chapter: 

•  Task  Description  and  Set  Up 

•  Examiner  Instructions  and  Script 

•  Score  Sheet 

•  Scoring  Guide 

•  Materials 


Guidance  for  Use  of  the  AMMP  Manual 

Administration  of  the  AMMP  requires  that  examiners  understand  the  theoretical  foundation  of 
the  test  tasks  and  are  competent  in  conducting  and  scoring  the  test  tasks. 

•  To  understand  the  theoretical  foundations  of  the  AMMP,  examiners  should  be  familiar 
with  papers  that  have  been  published  on  this  test  (e.g.,  Radomski  et  al.,  2013;  Scherer  et 
al.,  2013;  Smith  et  ah,  2014). 

•  To  assure  competence,  examiners  should  first  read  the  AMMP  Manual  in  its  entirety. 
Next,  examiners  should  watch  the  AMMP  Training  Modules.  After  doing  so,  at  least 
three  practice  administrations  of  each  test  task  are  recommended  before  administering  the 


test  task  on  a  subject  or  patient.  This  will  enable  the  examiner  to  become  comfortable 
simultaneously  reading  the  administration  script,  managing  any  equipment,  and 
observing  performance. 
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CHARGE  OF  QUARTERS  DUTY 


Charge  of  Quarters  Duty 
Task  Description  and  Set  Up 


Description 

The  SM  is  challenged  to  develop  and  execute  a  work  plan  for  completing  an  array  of  interleaving 
tasks  associated  with  his/her  hypothetical  assignment  to  CQ  duty. 

Purpose 

This  multi-task  provides  an  opportunity  to  observe  and  quantify  planning  and  implementing  a 
plan;  specifically,  how  a  SM  with  mTBI  approaches  an  unstructured  complex  task  when  only 
task  parameters  and  outcome  are  specified. 

mTBI-related  task  challenges:  Primary  •  Secondary  o 


Cognitive 

Sensory 

Physical 

Executive 

function 

Memory 

Attention 

Reaction 

time 

Eye  gaze 
tracking 

Scanning 

Vestibular 

Balance 

Exertion 

Bend- 

lift 

Manual 

UE 

Speed 

• 

o 

o 

o 

o 

Source 

Adapted  from  Multiple  Errands  Test-Simplified  Version  (Alderman  et  al.,  2003) 

Alderman,  N.,  Burgess,  P.W.,  Knight,  C.,  &  Henman,  C.  (2003).  Ecological  validity  of  a 
simplified  version  of  the  multiple  errands  test.  Journal  of  the  International  Neuropsychological 
Society,  9,  31-44. 

Materials  and  Supplies 

Set  up  and  administration  items: 

•  Blue  painters  tape 

•  Tape-measure 

•  Clipboard 

•  Administration  manual  and  scoresheet 

•  Stopwatch 

•  Pencils 

Test  task  items: 

Laminated  signs  to  be  posted  in  each  work  area  (Assembly  Area,  Supply  Closet,  Bulletin 
Board,  CQ  Desk) 

Wall  clock 
Assembly  area  - 

•  Table 

Examiner  sits  or  stands  in  a  location  to  fully  observe  the  table  in  order  to  observe  rule 
adherence  regarding  number  of  PVC  parts  in  the  Assembly  Area  at  any  given  point  in 
time. 


Supply  closet 

•  Basket  to  carry  items 

•  5-drawer  unit  for  footstool  parts/tools  with  drawers  labeled  as  follows  - 

o  Drawer/Bin  1-1”  diameter  PVC  pipe  (4,  12”  in  length;  6,  8”  in  length;  10,  4.5” 
in  length)* 

o  Drawer/Bin  2  -  Elbow  and  T-Connectors  (6,  1”  diameter  90  degree  elbow  PVC 
connectors;  7,  1”  diameter  T-  PVC  connectors) 
o  Drawer/Bin  3  -  End  Caps  (4,  1”  diameter  external  PVC  endcaps) 
o  Drawer/Bin  4  -  3-Way  Connectors  (8,  1”  diameter  3-way  elbow  PVC  connectors) 
o  Drawer  5  -  Other  (sandpaper,  masking  tape,  tape  measure,  labels,  timer) 

•  Fully  Stocked  Inventory  List  sign  posted  over  the  5-drawer  unit 

•  Table  on  which  to  place  5-drawer  unit  is  optional 

The  footstool  inventory  list  is  affixed  to  the  side  of  the  drawer  unit  so  that  it  is  not  be  visible 
while  the  subject  is  standing  in  any  of  the  other  work  zones. 

*PVC  pipe,  connectors  may  be  purchased  at  local  home  improvement  stores  or  ordered  directly 
from  Formufit,  Inc.  http://www.formufit.com/ 

Bulletin  board 

Signs  posted  on  the  wall:  barrack  lay-out,  CQ  duty  roster,  monthly  calendar,  diagram  for 
foot  stool  assembly,  2  foils  (information  not  relevant  to  the  task)  (see  Figure  5  for  layout).  Blue 
tape  must  extend  for  the  length  of  the  posted  materials  and  be  ~  4  feet  from  the  wall  (to  prevent 
subjects  from  reading  the  contents  from  outside  the  designated  area).  Select  a  space  in  which 
information  posted  in  the  Bulletin  Board  area  cannot  be  read  while  the  subject  is  standing 
in  any  of  the  other  work  zones. 

CQ  desk 

Table  or  desk 
Chair 

2  walkie  talkies** 

Basket  for  walkie  talkies 

Plastic  hanging  file  box  with  10  file  folders  labeled  as  follows: 

Blank  CQ  Duty  Reports 
Completed  CQ  Duty  Reports 
Inventory  Forms 
Completed  Inventory  Forms 
Incidence  Reports 
Blank  CQ  Duty  schedules 
Past  CQ  schedules  (past  4  months) 

Emergency  contacts 
2  blank  folders 

Laminated  Contact  List  (placed  such  that  it  is  visible/upright  in  the  File  box) 

Laminated  Communications  Roster  (positioned  on  desk  top) 

Laminated  Walkie  Talkie  instructions  (positioned  on  desk  top)** 


/  u 


**Walkie  talkies  may  be  purchased  at  local  home  improvement  stores.  Laminated  Walkie  Talkie 
instructions  include  in  the  Manual  pertain  to  the  Cobra  MicroTalk  made  by  Cobra  Electronics. 
Laminated  instructions  may  need  to  be  modified  for  other  models. 
https://www.cobra.com/products 

Test  Task  Set  Up 

Space  estimate:  Approximately  6  feet  by  1 1  feet  area 

The  CQ  Duty  test  task  can  be  set  up  in  whatever  configuration  aligns  with  the  available  clinical 
or  testing  space  (Figure  1).  Work  area  boundaries  are  established  by  the  placement  of  blue 
painter’s  tape.  The  measurements  in  Figures  2  -  5  are  estimates.  Testers  should  use  the  Checklist 
below  to  make  sure  that  tape  placement  assures  that  SM  must  step  into  the  work  area  in  order  to 
complete  relevant  test  tasks. 

Figure  1.  Example  of  how  the  CQ  Duty  test  task  might  Figure  2.  CQ  Desk 
be  set  up  in  a  clinical  space. 


Figure  3.  Assembly  area 


Figure  4.  Supply  closet 


Figure  5.  Bulletin  board 


Test  Task  Set  Up  Checklist: 

_ Is  the  wall  clock  visible  from  all  work  areas? 

_ Are  the  CQ  duty  schedule,  barrack  layout  readable  ONLY  if  standing  inside  the  blue  tape 

in  the  Bulletin  Board  work  area?  [If  one  or  more  can  be  read  from  outside  the  blue  tape,  move 
the  tape  further  out.] 

_ Is  it  impossible  to  view  the  footstool  diagram  (Bulletin  Board)  from  the  Assembly  Area? 

[Select  a  location  for  the  Bulletin  Board  in  which  SM  cannot  see  the  diagram  from  the  Assembly 
Area.] 

_ _  Is  the  Fully  Stocked  Inventory  List  readable  ONLY  if  standing  inside  the  blue  tape  in  the 

Supply  Closet  work  area?  [If  the  sign  can  be  from  outside  the  blue  taped  area,  move  the  tape 
further  out.] 

_ Is  the  Contact  List  and  Communications  Roster  readable  ONLY  if  standing  inside  the  blue 

tape  of  the  CQ  Duty  Desk  work  area?  [If  one  or  more  can  be  read  from  outside  the  blue  tape, 
move  the  tape  further  out.] 


Charge  of  Quarters  Duty 
Examiner  Instructions  and  Script 

Before  instructing  the  participant,  turn  on  participant’s  walkie  talkie  and  set  to  radio 
frequency  6  and  place  in  the  basket  at  the  CQ  desk;  turn  on  the  examiner’s  walkie  talkie 
and  set  to  radio  frequency  10.  Make  sure  a  clock  can  be  seen  from  all  areas.  During 
the  task,  sit  or  stand  in  a  location  that  allows  you  to  fully  and  easily  observe  the  table  in 
the  Assembly  Area  (AA)  in  order  to  observe  rule  adherence  regarding  #  of  PVC  parts  in 
the  AA  at  any  given  point  in  time. 

INTRODUCTION 

Provide  participant  with  CQ  Duty  Report  affixed  to  clipboard  and  state  the  following: 

During  this  exercise,  you  must  complete  some  assignments  while  pulling  CQ 
duty.  This  is  your  copy  of  the  CQ  Duty  Report.  It  describes  task  instructions, 
rules,  and  includes  places  for  you  to  write  notes.  You  can  use  it  throughout  the 
exercise  along  with  whatever  devices  and  techniques  that  you  think  will  help  you 
perform  at  your  best.  I  will  orient  you  to  your  assignments,  the  test  spaces  and 
the  rules  and  you  can  ask  questions. 

INSTRUCTIONS 

A.  Description  of  assignments 

Follow  along  as  I  describe  your  assignments. 

•  Radio  SPC  Smith  at  the  Guard  Shack  to  report  that  you  are  taking  over  CQ 
duty. 

•  Assemble  a  PVC  footstool  for  CPT  James. 

•  After  exactly  5  minutes,  stop  what  you  are  doing  and  radio  the  First 
Sergeant  to  report  the  number  of  vacant  barracks.  Also  ask  if  there  are  any 
additional  tasks  to  be  completed  while  on  duty.  If  so,  these  must  be  added 
to  your  assignment  list. 

•  On  your  CQ  Duty  Report,  write  down  the  following  4  items: 

a)  number  of  3-way  PVC  connectors  remaining  in  stock  after  the 
footstool  is  assembled; 

b)  Formufit’s  address  (manufacturer  of  the  footstool  materials); 

c)  the  telephone  number  of  the  Service  Member  who  is  scheduled  for 
CQ  duty  next  Wednesday; 

d)  the  room  #  that  PVT  Sullivan  is  in. 

•  Write  legibly.  You  will  not  receive  full  points  if  I  cannot  read  your 
handwriting. 

•  Return  all  supplies  and  materials  to  their  original  locations  and  place  the 
footstool  on  the  CQ  desk  (touch  the  desk  surface)  at  the  end  of  the  exercise. 

•  File  your  completed  CQ  Duty  Report  in  the  folder  entitled,  Completed  CQ 
Duty  Reports. 


•  After  completing  all  of  your  tasks,  radio  SPC  Smith  at  the  Guard  Shack  and 
report  that  you’ve  completed  your  assignments. 

Next,  I  am  going  to  point  out  your  Work  Areas  for  this  task.  But  first,  do  you 
have  any  general  questions  about  your  assignment  so  far?  If  yes,  clarify.  If  no, 
begin  orientation  to  testing  space. 

B.  Orient  participant  to  the  testing  space. 

I  will  now  orient  you  to  the  testing  space.  Everything  you  need  to  complete 
this  exercise  is  located  in  the  4  work  areas  marked  with  blue  tape. 

B1  .Walk  to  the  areas  with  the  participant  to  point  out  location  of  materials  and  area 

perimeter.  Start  by  walking  to  the  CQ  Desk. 

•  We  are  currently  at  the  CQ  Desk. 

•  Use  this  walkie  talkie  during  the  exercise.  Push  this  button  (point)  to  speak 
into  the  walkie  talkie;  push  this  button  (point)  to  change  radio  frequency 
channels.  Use  these  instructions  if  you  need  more  help  (point). 

•  Use  this  Communication  Roster  (point)  to  locate  the  correct  radio  frequency 
channels  for  various  personnel  that  you  need  to  contact  during  the  task. 

•  Obtain  and  file  required  forms  here  (point  to  file  box).  Also  note  the  pad  of 
paper  and  pencil  and  a  Contact  List  for  those  assigned  to  CQ  duty  (point  to 
list  that  sticks  out  of  last  folder). 

B2.  Walk  over  to  Bulletin  Board. 

•  Here  is  the  Bulletin  Board.  Here  you  will  find  information  that  you  need  to 
report  to  command  including  a  map  of  the  barracks,  a  CQ  duty  schedule,  a 
clock,  and  a  diagram  regarding  how  to  assemble  the  footstool.  Note  that 
this  diagram  is  the  only  information  you  have  about  how  to  construct  the 
footstool. 

B3.  Walk  to  the  Supply  Closet. 

•  This  area  is  the  Supply  Closet.  All  the  materials  you  will  need  for  building 
the  footrest  are  located  in  this  area.  Each  drawer  is  labeled  but  keep  in 
mind  that  there  are  some  parts  in  the  drawers  that  you  do  not  need  for  the 
project.  You  may  use  any  of  the  items  in  the  last  drawer  (point  to  drawer 
labeled  Other).  Here  is  an  inventory  list  that  specifies  how  many  parts  we 
need  to  keep  on  hand  and  the  address  of  Formufit  to  use  for  re-ordering 
supplies  (point  to  list).  Additionally,  there  is  a  basket  you  may  use  to  carry 
items  from  one  area  to  another. 

B4.  Walk  to  the  Assembly  Area. 

We  are  now  in  the  Assembly  Area.  Assemble  the  footrest  only  in  this  area. 

Do  you  have  any  questions  about  the  work  spaces?  (If  yes,  clarify.  If  no, 
continue.) 


C.  Review  the  task  rules. 


I  will  now  explain  the  task  rules.  During  this  exercise,  you  must  follow  these 

rules  (point  out  rules  on  CQ  Duty  Report). 

•  You  must  carry  out  all  of  these  tasks  but  may  do  so  in  any  order. 

•  Assemble  the  footrest  only  in  the  Assembly  Area. 

•  Bring  only  the  number  of  PVC  parts  needed  for  the  footrest  from  the  supply 
closet  to  the  assembly  area. 

•  Do  not  move  or  remove  any  of  the  wall  signs  during  the  course  of  this 
exercise. 

•  In  order  to  score  the  most  points,  your  trips  between  work  areas  should  be 
kept  to  a  minimum.  You  should  return  to  an  area  only  if  it  is  absolutely 
necessary  to  complete  the  task  or  follow  the  instructions  correctly.  You 
will  get  the  most  points  if  you  can  complete  this  exercise  in  7  trips. 

•  Do  this  exercise  as  quickly  as  possible  without  rushing. 

•  You  may  not  ask  questions  for  further  guidance  about  this  exercise  once 
the  test  starts. 

Do  you  understand  these  rules?  If  no,  clarify.  If  yes,  continue. 


EXAMINER  GUIDANCE: 

If  the  SM  asks  whether  he/she  can  use  his  alarm,  phone,  watch  during  the 
test,  state:  “Use  any  strategy  or  device  that  you  think  will  help  you  do 
your  best.” 

If  the  SM  asks  any  questions  about  the  test  task  and/or  procedures  BEFORE 
the  test  begins,  answer  the  questions  directly. 


D.  Verify  that  participant  understands  the  instructions.  Move  outside  of  the  designated 

work  areas  into  a  neutral  zone.  Using  the  CQ  Duty  Report,  the  Participant  restates  the 

task  instructions  and  rules. 


Now  brief  me  on  what  you  are  being  asked  to  do.  Fill  in  any  gaps  that  the 
participant  may  not  have  included. 

After  doing  so  and  answering  all  questions  about  the  task,  place  the  subject  on  the  start 

X  outside  of  all  taped  areas.  State:  Remember,  you  must  complete  all  of  the 
assigned  tasks  but  may  do  so  in  any  order.  Start  and  begin  the  timer. 

[Participant  begins  the  exercise.] 


EXAMINER  GUIDANCE:  IF  THE  SUBJECT  IS  USING  THE  WRONG  RF 

If  the  SM  tries  to  perform  step  1  (radioing  Guard  Shack)  but  is  not  on  the  correct 
frequency,  state:  I  can’t  hear  you  on  the  radio. 

If  the  SM  still  does  not  figure  out  to  change  the  rf,  state:  you  are  on  the  wrong  rf. 
See  side  2  of  score  sheet  for  scoring  instructions. 


EXAMINER  GUIDANCE:  HOW  TO  RESPOND  TO  QUESTIONS  ASKED  DURING 
TESTING  AND  SCORE  RULE  BREAKS  RE  QUESTIONS 

-  Scoring:  Every  question  asked  =  1  rule  break. 

If  a  subject  asks  a  question  during  the  test,  state:  “Do  what  you  think  is  best.” 

If  the  subject  has  not  figured  out  a  solution  to  his/her  own  question  within  1  minute, 
point  to  the  relevant  information  on  his  or  her  CQ  Duty  report  and  provide  information  to 
get  him  or  her  back  on  track.  [Scoring:  The  item  for  which  the  subject  needs  this 
examiner  assistance  is  scored  a  0.] 


E.  Re-set  examiner  walkie  talkie  during  task. 

At  the  beginning  of  the  test,  the  examiner’s  walkie  talkie  is  set  at  frequency  10 
(corresponding  to  Gate  Guards/Guard  Shack). 


EXAMINER  GUIDANCE:  WALKIE  TALKIE  RESPONSE  TO  REPORT  TO  GUARD 
SHACK 

Participant:  “SPC  Smith  (or  Guard  Shack)  this  is  XXX,  Over” 

Guard  Shack:  “Xxx  this  is  SPC  Smith,  go  ahead  over” 

Participant:  “Beginning  CQ  duty” 

Guard  Shack:  “Good  copy,  over.”  Or  “Roger  that,  Over.” 

Participant:  “Over” 


After  the  subject  radios  to  check  in  (task  1  on  score  sheet),  the  examiner  changes  the 
radio  to  frequency  5  (corresponding  to  First  Sergeant  and  task  4  on  score  sheet). 

[5  MINUTES  INTO  TASK] 

F.  Provide  additional  instructions  when  participant  radios  in  after  5  minutes  and  asks 

about  more  tasks. 

•  Get  an  Inventory  Form  from  the  files  at  the  CQ  desk. 

•  Inventory  PVC  supplies  in  SC  using  the  form  based  on  supply  status 
once  foot  rest  is  assembled. 

•  Once  this  is  done,  file  the  order  form  in  the  “Completed  Inventory 
Forms”  folder  at  the  CQ  desk. 

EXAMINER  GUIDANCE:  WALKIE  TALKIE  DIALOGUE  WITH  FIRST  SERGEANT 

Participant:  “First  Sergeant  this  is  XXX,  Over” 

First  Sergeant:  This  is  First  Sergeant.  Over. 

Participant:  Reporting  vacant  barracks  as  9.  Any  additional  assignments  over? 

First  Sergeant: 

1 .  Get  an  Inventory  Form  from  the  files  at  the  CQ  desk. 

2.  After  you  complete  the  foot  rest,  inventory  PVC  supplies  in  SC  using  the 
Inventory  Form. 

3.  File  the  form  in  “completed  inventory  forms”  folder  at  CQ  desk,  how  copy?” 

Participant:  Repeats  instructions,  clarification  made  if  error.  If  correct,  “Good  copy,  Over” 


After  the  subject  completes  task  4,  the  examiner  resets  the  walkie  talkie  to  frequency 
10  (corresponding  to  task  18  on  score  sheet). 

G.  The  task  is  discontinued  when: 

a)  The  Participant  radio  reports  to  the  Guard  Shack  that  he/she  has  finished  the 
assignments  OR; 

b)  The  Participant  states  he/she  does  not  want  to  continue  OR; 

c)  Participant  demonstrates  behavioral  contraindications  as  specified  in  IRB  application 
OR; 

d)  The  Participant  is  still  performing  the  task  at  30  minutes  and  does  not  appear  to  be 
within  2  minutes  of  completion. 


Rev.  10-15-13;  11-26-13;  12-2-13;  2-4-14;  4-1- 
14;  7-15-14;  9-11-14;  6-2015 


Study  ID: 

Rater: 

Date: 

Order: 

Did  SM  complete  test  task? _ Yes  _ No  (Examiner  stopped)  _ No  (Subject  requested  to  stop) 

CQ  Duty  Score  Sheet 


Tasks 

2=  100%  accurate,  no  cues  required 

1=  Completed  but  only  partly  correct  OR  1  =  Required  cues  (1,  3,  4,  6,  13 
only*) 

Task 

Score 

0,1,2 

Sub¬ 

totals 

Scratch  Pad  for 
visits  to  WORK 
AREAS 

1-Radio  SPC  Smith/Guard  Shack  and....[rf  10]  STATE *:  1  can’t  hear  you  on  the 
radio. 

/10 

SC,  AA,  BB,  CQ 
Start  (outside 
work  areas) . . . 

1  .To 

2. To 

3. To 

4. To 

5. To 

2  -Report  that  you  are  taking  over  CQ  duty. 

3-  Between  4  min  30  sec  -5  min  30  sec,  participant  stops  what  he/she  is  doing* 

....  NOTE:  IF  THE  SM  DOES  NOT  STOP  WHAT  HE/SHE  IS  DOING  BY  6  MINUTES  & 

30  SEC,  STATE*:  Stop  and  do  what  1  instructed  you  to  do  5  minutes  into  the  task. 

4-  Radio  1SG*  [rf  5]  CUE :  /  can’t  hear  you  on  the  radio. 

NOTE:  IF  THE  SM  DOES  NOT  KNOW  WHAT  TO  DO  AT  THE  5  MINUTE  INTERVAL, 
STATE*:  Check  your  CQ  duty  report  to  see  what  you  are  supposed  to  do  now. 

5-  Report  #  of  vacant  barracks  rooms.  [9] 

6. To 

7. To 

8. To 

Q  Tn 

6-  Ask  if  there  are  any  additional  tasks  to  be  completed  while  on  duty*. 

NOTE:  IF  THE  SM  DOES  NOT  ASK  THE  QUESTION,  STATE*: 

What  else  are  you  supposed  to  ask  me? 

/8 

7-  Get  an  Inventory  Form  from  the  files  at  the  CQ  desk. 

O.  1  u 

lO.To 

8-  Assemble  PVC  footrest 

9-  Put  the  footrest  on  the  CQ  desk  before  completing  the  exercise. 

1 1  .To 

12. To 

13. TO 

10-  Inventory  PVC  supplies  in  SC  using  the  form  based  on  supply  status 
once  foot  rest  is  assembled. 

no 

1 1  -  Files  Inventory  Form  in  “Completed  Inventory  Forms  folder” 

14.To 

12-Return  all  supplies  and  materials  to  their  original  locations  before  completing 
the  exercise. 

15. TO 

16. T0 

17. To 

13-  Radio  SPC  Smith/Guard  Shack  ....  [rf  10]  STATE*:  1  can’t  hear  you  on  the 
radio. 

14-  Report  task  completion. 

18. T0 

19. TO 

20. To 

15-  Write  down  telephone  #  of  SM  scheduled  next  week  at  this  time 
[Anderson  @  703-555-5564] 

no 

16  Write  down  the  #  of  3-way  PVC  elbows  in  stock  after  assembling  [6] 

21 .  To 

17-  Write  down  the  mailing  address  of  Formufit  Inc.  [15954  S.  Mur  Len  Road 
#311  Olathe,  KS  66602] 

22.  To 

23.  To 

24.  To 

18-  Write  down  room  #  that  PVT  Sullivan  is  in  [308] 

19-  File  completed  CQ  Report  in  “Completed  CQ  Duty  Reports”  folder 

25.  To 

Task  performance  total  score 

00 

00 

< 

26.  To 

27.  To 

Rules  (see  rule  break  definitions  on  back) 

Rule 

break 

(Y/N) 

#  of 
rule 
breaks 

28.  To 

29.  To 

30.  To 

31.  To 

Do  not  ask  questions  for  further  guidance  about  this  exercise  once  the  test 
starts. 

Assemble  the  footrest  only  in  AA. 

32.  To 

33.  To 

Bring  only  the  number  of  PVC  parts  needed  for  the  footrest  to  the 

Assembly  area  [22  parts]. 

34.  To 

35.  To 

Do  not  remove  any  of  the  signs  from  the  walls  of  the  work  areas. 

D.  Total  #  of 
visits  to 
complete 
exercise: 

Performance 

time: 

E.  min  sec 

B. 

C. 

Evervthina  else  must  be  scored  in  real  time  EXCEPT  10. 11. 15-19 


Visit  =  any  body  part  crosses  into  taped  work 
area. 


Rules 

Rule  break  examples 

Do  not  ask  questions  for  further  guidance  about  this 
exercise  once  the  test  starts. 

1  rule  break  for  every  question  asked.  Making  statements  aloud  t 
rule  break.  [“Can  you  help  me  get  this  walkie  talkie  to  work?  What  is 
the  correct  RF?”  =  2  rule  breaks] 

Assemble  the  footrest  only  in  AA. 

Each  time  SM  puts  2  parts  together  outside  of  AA  =  1  rule  break 

SM  connects  PVC  elbow  to  4.5”  piece  in  SC  =  1  rule  break 

Bring  only  the  #  of  PVC  parts  that  are  needed  for  the 
footrest  to  the  AA  [only  22  PVC  parts  in  AA  at  any  point! 

If  SM  has  24  PVC  parts  at  AA  =  2  rule  breaks 

SM  brings  masking  tape  to  AA  =  0  rule  break  (not  a  PVC  part) 

Tasks 

Scoring  examples  (1, 0) 

1.  Radio  SPC  Smith/Guard  Shack  and...  [rf  10] 

1  =  radios  Smith/Guard  Shack  after  cue  re  rf 

0=  does  not  do  this  task  at  all  OR  requires  further  cueing  re  rf 

2.  Reports  taking  over  CQ  duty 

1=  reports  something  other  than  that  he/she  is  taking  over  duty 

0=does  not  do  this  task  at  all 

3.  Between  4  min  30  sec  -5  min  30  sec,  participant  stops 
what  he/she  is  doing  .... 

1=  stops  what  he/she  is  doing  between  5  min  30  seconds  and  6  min 

30  seconds 

0=  called  before  4  min  30  sec  OR  didn’t  radio  in  by  6  min  30  sec  OR 
examiner  instructs  to  stop  what  he/she  is  doing  now 

4.  Radios  1SG  [rf  5] 

1  =  radios  1  SG  after  cue  re  rf 

1  =  radios  after  being  cued  by  examiner  to  check  CQ  Report 

0  =  even  with  cue,  SM  doesn’t  know  to  radio  1SG  and  is  instructed  to 
do  so 

5.  Report  #  of  vacant  barracks  rooms.  [9] 

1  =  reports  incorrect  #  of  vacant  barracks 

0=  doesn’t  report  this  information  at  all 

6.  Ask  if  there  are  any  additional  tasks  to  be  completed 
while  on  duty*. 

1  =asks  if  there  are  additional  tasks  to  be  completed  after  being  cued 

0=  even  with  cue,  SM  does  not  ask  the  question  and  the  examiner 
simply  provides  the  3  additional  instructions  (7,10,  11) 

7.  Obtain  an  inventory  form  from  the  files  at  CQ  desk 

1  =obtains  the  wrong  form  from  CQ  desk 

0=  doesn’t  get  any  form  from  CQ  desk 

8.  Assemble  PVC  footrest 

1  =  constructs  footrest  but  made  errors  related  to  1  -  2  parts 

0  =  does  not  do  this  task  at  all  OR  attempted  with  errors  on  3  or  more 
parts 

9.  Put  the  footrest  on  the  CQ  desk  before  completing  the 
exercise. 

1  =puts  the  footrest  on  a  table  or  desk  other  than  the  CQ  desk 

0  =  puts  footrest  on  the  floor 

1 0.  Inventory  supplies  remaining  once  the  footrest  is 
assembled 

1  =  takes  inventory  but  some  of  the  values  are  incorrect 

1  =takes  inventory  but  answers  are  not  legible 

0=  does  not  take  inventory 

1 1 .  Files  Inventory  Form  in  “Completed  Form ”  folder 

1=files  the  form  but  in  the  wrong  folder  OR  form  in  hanging  file  but  not 
in  folder 

0=  does  not  file  the  form 

12.  Returns  all  supplies  and  materials  to  their  original 
locations  before  completing  the  exercise. 

1=returns  some  but  not  all  supplies  and  materials  to  original  locations 

0=  does  not  return  any  of  the  supplies  and  materials  to  their  original 
locations  before  radioing  that  he/she  is  done  with  CQ  duty 

13.  Radio  SPC  Smith/Guard  Shack,  [rf  10] 

1  =  radios  Guard  Shack  after  cue  re  rf 

0=  does  not  do  this  task  at  all  OR  requires  further  cueing  re  rf 

14.  ...to  report  task  completion. 

1=reoorts  something  other  than  task  completion  OR  calls  in  before 
completing  task  (ie  filing  CQ  duty  report  as  instructed) 

0=does  not  do  this  task  at  all 

15.  Write  down  telephone  #  of  SM  scheduled  next 
week  at  this  time 
[Anderson  <§>  703-555-5564] 

1  =  fills  in  name  of  Anderson  but  not  phone  # 

1  =  fills  in  wrong  phone  number 

1=fills  out  form  but  answers  are  not  legible 

0  =  does  not  fill  in  this  part  of  CQ  duty  report 

16.  Write  down  the  #  of  3-way  PVC  elbows  in  stock 
after  assembling  foot  rest  [6] 

1  =fills  in  the  wrong  #  of  3-way  PVC  elbows 

1  identifies  #  but  answers  are  not  legible 

0=  does  not  fill  in  this  part  of  the  CQ  duty  report 

18.  Write  down  the  room  #  for  PVT  Sullivan  [308] 

1  =fills  in  this  part  of  CQ  Report  but  with  wrong  room  # 

1  identifies  room  #  but  not  clearly  legible 

0=does  not  fill  in  this  part  of  CQ  Report 

19.  File  completed  CQ  Report  in  the  “Completed  CQ 
Duty  Reports”  folder 

1=files  the  form  but  in  the  wrong  folder  OR  form  in  hanging  file  but  not 
in  folder 

0=  does  not  file  the  form 

Task  performance  is  over  immediately  after  the  subject  radios  Guard  Shack  and  reports  task  completion.  Stop  timer 
and  record  the  performance  time.  SM  scores  a  “0”  for  any  task  completed  after  radioing  Guard  Shack  to  report 
task  completion. 


estion  asked.  Making  statements 
you  help  me  get  this  walkie  talkie  to 
:  RF?”  =  2  rule  breaks] 


Charge  of  Quarters  Duty 
Scoring  Guide 


Examiner  scoring  supplies/materials: 

•  1  stopwatch 

•  Clipboard 

•  Pencil 

•  Subject  score  sheets 

•  Radio  /  walkie  talkie 

Definitions  of  key  underlying  concepts: 

Rules  -  Instructions  that  specify  HOW  a  task  test  is  to  be  completed  which  could 
be  broken  >  1  time  during  test-task  performance.  These  rules  may  be  adhered  to 
or  broken. 

Task  performance  -The  extent  to  which  the  subject  independently  and  accurately 
completed  each  task  element  as  instructed. 

Performance  time  -  The  number  of  minutes  and  seconds  between  when  the 
examiner  says,  “Start”  and  when  the  participant  a)  reports  to  SPC  Smith/Guard 
Shack  that  he/she  is  finished  with  the  task  OR  b)  reports  that  he/she  does  not 
want  to  continue. 

Visit  -  A  visit  occurs  whenever  any  body  part  crosses  into  a  taped  Work  Area. 
Visits  are  an  observable  metric  for  work  efficiency. 


Scoring  procedures  for  performance  subscores: 

Before  starting  the  task 
Fill  out  the  following: 

-  Subject’s  study  ID 

-  Your  Rater  ID 

-  Today’s  date 

-  Where  in  the  test  order  the  subject  is  performing  this  test-task 
At  task  start 


Performance 

dimension 

Scoring  procedures 

Performance 

subscore 

Performance  time 

The  examiner  starts  stopwatch  when  all  task  questions  have  been 
answered  and  immediately  after  he/she  states,  “Start”. 

After  starting  the  task,  the  examiner  does  not  cue  the  subject  or 
answer  questions  (see  Rule  1). 

(See  “E”  below) 

During  task 


Performance 

dimension 

Scoring  procedures 

Performance 

subscores** 

A.  Task 
performance 

There  are  19  tasks  listed  on  the  score  sheet. 

All  task  must  be  scored  in  real-time  as  the  subject  performs  the  task 
except  10,  11,  15-19  (which  mav  be  scored  after  the  subiect  has 
completed  the  task  but  before  set  up  for  the  next  participant). 

The  examiner  assigns  a  0,  1 ,  or  2  for  each  of  the  1 9  tasks  based  on 
observations  of  subject  performance. 

A  =  Task 
performance  total 
score 

Scores  are 
summed  for  the 

19  tasks  and 
recorded  on  the 
score  sheet. 

A=  /38 

possible  points 

2  =  1 00%  accurate,  no  cues  required 

1=  Completed  but  only  partly  correct  OR  1  =  Required  cues  (1, 3,  4,  6, 

13  only) 

0  =  Did  not  complete  or  perform 

Note:  Subiects  are  not  cued  durina  task  performance  except  for  tasks 

1, 3,  4,  6,  13. 

Refer  to  Side  2  of  the  score  sheet  for  examples  of  performance 
warranting  scores  of  0  or  1  for  each  of  the  19  tasks. 

B.  Rule  breaks 

There  are  4  rules: 

•  Do  not  ask  questions  for  further  guidance  about  this  exercise 
once  the  test  starts. 

•  Assemble  the  footrest  only  in  the  AA. 

•  Bring  only  the  number  of  PVC  parts  needed  for  the  footrest  to  the 
AA  [22  parts]. 

•  Do  not  remove  any  of  the  signs  from  the  walls  of  the  Work  Areas. 

If  the  subject  breaks  rule  #  1 ,  a  Y  is  placed  in  the  corresponding  “Rule 
break”  column  of  the  score  sheet.  If  he/she  does  not  break  the  rule  during 
any  part  of  the  task,  an  N  is  placed  in  that  column. 

Same  for  rule  #  2,  3,  and  4. 

Refer  to  Side  2  of  the  score  sheet  for  examples  of  commonly  broken 
rules. 

B  =  Total  #  of 
rules  broken 

Sum  the  #  of  Ys 
in  the  Rule  break 
column 

B=  /4 

possible  rules 
broken 

C.  #  of  rule 
breaks 

Each  time  a  rule  is  broken,  the  examiner  puts  a  check-mark  in  the 
corresponding  column  labeled  “#  of  rule  breaks”. 

For  example:  If  the  subject  asks  the  examiner  4  questions  during  task 
performance,  there  would  be  4  check-  marks  in  “#  of  rule  breaks”  column 
for  rule  #  1. 

C  =  Total 
frequency  of  rule 
breaks 

For  each  rule 
broken,  the 
examiner  counts 
the  #  of  check¬ 
marks  and 
records  the  total 
in  the 

corresponding  “# 
of  rule  breaks” 
column.  Next, 
the  examiner 
sums  these 

D.  Task 
organization, 
planning, 
efficiency 


The  subject  begins  the  task  positioned  in  the  Neutral  Zone  (outside  of  the 
designated  work  areas). 

The  examiner  sequentially  writes  down  each  Work  Area  the  subject 
enters  throughout  the  task  (see  below),  beginning  with  the  first  Work  Area 
the  subject  enters  after  the  examiner  instructs  him  or  her  to  “start”. 


columns  to 
determine  C. 

(total  frequency 
of  rule  breaks). 

D  =  Total  #  of 
visits  to  work 
areas  to 

complete  the  task 


A  visit  occurs  whenever  any  body  part 
crosses  into  a  taped  Work  Area. 

For  1 .  To _ write  the  abbreviation  of 

the  first  work  area  the  subject  enters  after  the 
examiner  says,  “start”. 

For  2.  To _ write  the  abbreviation  for 

the  work  area  that  the  subject  visits  next  (and 
so  on). 

Supply  closet  =  SC 
Assembly  area  =  AA 
Bulletin  board  =  BB 
CQ  desk  =  CQ 


Scratch  Pad  Pw 
vtaits  to  WORK 
AREA* 


SC,  AA,  BE,  CQ 
Start  (outs  de 
sr^asj . . . 

I. To  CGt 
2  To  ITS 
3-To  SC- 

J. To  A-A- 

5  To  fj  ft 

6  To 

7  Jo  C£L 
6. To 


At  task  completion 


Performance 

dimension 

Scoring  procedures 

Performance  subscore 

E.  Performance 
time 

When  the  subject  radios  SPC 

Smith/Guard  Shack  OR  he/she  states  that 
he/she  does  not  want  to  continue,  the 
examiner  stops  the  stopwatch  and 
records  the  length  of  time  for  task 
completion. 

(E)Total  task  performance  time  -  Time  in 
minutes/seconds  that  it  takes  for  the  subject 
to  complete  the  task  after  verifying/clarifying 
all  instructions  until  he/she  notifies  examiner 
of  task  completion. 

After  subject  has  completed  task 

If  you  have  not  done  so  already,  score  tasks  10,  11,  15-19  and  include  those 
scores  in  (A)  Task  performance  total  score. 

Also,  see  the  Answer  Sheet  for  the  Inventory  Form  for  correct  answers. 

CQ  Scoring  Frequently  Asked  Questions 
Task  Performance  (A) 

How  should  I  score  performance  if  the  subject  makes  errors  using  the  walkie 
talkie  (such  as  setting  it  on  the  wrong  radio  frequency)  ? 

If  the  SM  tries  to  perform  Task  1,4,  13  (radioing  Guard  Shack)  but  is  not 
on  the  correct  frequency,  state:  I  can’t  hear  you  on  the  radio.  [Score  1] 
If  the  SM  still  does  not  figure  out  to  change  the  rf,  state:  you  are  on  the 
wrong  rf.  [Score  0] 

Why  am  I  required  to  provide  cues  for  tasks  1,  3,  4,  6,  and  13  but  not  the  rest  of 
them? 


The  performance  of  each  of  these  tasks  is  essential  to  the 
completing  subsequent  steps  in  the  CQ  Duty.  Therefore,  cueing  may 
be  necessary  to  allow  the  subject  to  complete  the  entire  activity. 

What  if  I  can’t  read  the  subject’s  handwriting  on  the  CQ  Duty  Report  or  the 
Inventory  Form? 

During  your  instructions,  the  subject  was  told:  Write  legibly.  You  will  not 
receive  full-points  if  I  cannot  read  your  handwriting. 

Each  task  requiring  a  written  answer  (15, 16, 17, 18)  that  is  not  legible 
is  scored  a  1 . 

Rules  (B  &  C) 

What  if  the  subject  asks  me  a  question  after  the  test  begins?  What  should  I  say 
and  how  should  the  rule  break  be  scored? 

Every  question  asked  =  1  rule  break.  If  he/she  asks  a  question, 
state,  “do  what  you  think  best”  and  record  “Y”  under  the  Rule  break 
(Y/N)  column  and  a  slash  mark  for  every  instance  in  which  1  question 
is  asked. 

If  the  subject  has  not  figured  out  a  solution  to  his/her  own 
question  within  1  minute,  point  to  the  relevant  information  on  his  or 
her  CQ  Duty  report  and  provide  information  to  get  him  or  her  back  on 
track.  [Scoring:  The  item  for  which  the  subject  needs  this  examiner 
assistance  is  scored  a  0.] 


Rules  (see  rule  break  definitions  on  beck) 

Rule 

break 

(Y/N) 

of  rule 
brsefcs 

Do  not  ask  questions  for  further  guidance  about  this  exercise  once  the  test 
starts. 

V 

// 

Assemble  tbs  footrest  only  In  AA. 

fj 

l 

Bring  only  the  number  of  PVC  parts  needed  for  the  footrest  to  the 

Assombly  area  [22  parts). 

A/ 

0 

Do  not  remove  any  of  the  signs  from  the  waifs  of  the  work  areas. 

AJ 

D 

Performance  1  E-  mm  sec 
time: 

Is. 

i 

C  Tote 

How  will  I  know  how  many  PVC  parts  the  subject  brings  to  the  AA  ? 

Position  yourself  so  that  you  can  count  the  #  of  parts  on  the  table  as 
the  subject  works.  Or  observe  if  there  are  parts  on  the  table  leftover 
after  the  footstool  is  assembled.  For  each  part  in  excess  of  22  in  the 
AA,  the  subject  receives  1  rule  break. 


Rules  (in  rule  break  definitions  on  beck) 

Rule 

break 

(™) 

frequency 

0#  rule 
tweafce 

Oo  not  ask  questions  for  further  guidance  about  this  exercise  once  the  test 
starts 

A / 

0 

Assemble  the  footrest  only  in  AA 

ft 

0 

Bring  only  the  number  of  PVC  parts  needed  for  the  footrest  to  the 

Assembly  area  (22  parts). 

Y 

III 

Oo  not  remove  any  of  the  signs  from  the  wails  of  the  work  areas. 

N 

0  ' 

B 

Toni 

l 

C.  Total  • 

3  : 

Performance  1  E. _ min _ sec 

time: 

Visits  (D) 


Scratch  Pad 
for  visile  to 
WORK  ARFAs 

5C,  AAP  BB. 

ca 


Start  foutsids 
work  areas) .. 

1. T0  JLft 

2. TO 

3.  To  frft 

4. Tti  Tf> 

5. To  A  A- 

6. T0  A~AJ 


l/W?af  if  the  subject  starts  to  leave  a  Work  Area  (with  one  foot  inside  the  taped 
area  and  one  foot  outside)  but  remembers  something  else  to  do,  turns  around  so 
that  both  feet  are  now  in  the  Work  Area? 

This  person  never  left  the  Work  Area  -  no  “visits”  are  recorded  for 
setting  1  foot  outside  of  the  taped  area. 


What  if  the  subject  is  able  to  reach  into  the  drawers  of  the  SC  to  get  PVC  parts 
without  stepping  across  the  tape  into  the  SC  Work  Area? 

A  visit  occurs  any  time  a  body  part  crosses  the  tape  into  a  Work  Area 
and  so  reaching  into  the  SC  constitutes  a  visit. 


What  if  one  of  the  PVC  parts  falls  off  the  table  and  rolls  outside  the  AA  and  the 
SM  goes  to  retrieve  it?  Does  this  count  as  a  “visit”  when  he  walks  back  into  the 
AA  to  continue  assembling  the  footrest? 

Yes.  Every  time  a  SM  crosses  the  tape  into  a  Work  Area,  a  visit  is 
recorded  -  regardless  of  the  reason  for  doing  so. 


Performance  Time  (E) 

What  if  the  subject  radios  the  Guard  Shack  to  report  task  completion  and  then 
files  the  CQ  Duty  report  at  the  CQ  desk?  At  what  point  do  I  stop  the  stopwatch  to 
record  performance  time? 

Stop  the  stopwatch  immediately  after  he/she  reports  that  he/she 
completed  the  task  and  record  as  the  performance  time.  Subject  scores  a 
“0”  for  item  9,  (Files  Inventory  Form  in  ‘‘Completed  Inventory  Forms” 
folder)  because  it  was  not  performed  during  the  task. 


ANSWER  SHEET  FOR  EXAMINERS 
[Parts  inventory  once  the  footrest  has  been  assembled] 


Inventory  Form 

Item 

Current  Stock 

Needed 

3  way  Elbow  PVC 
Connectors 

6 

2 

T-  PVC 

Connectors 

3 

4 

90  degree  Elbow 
Connectors 

4 

2 

External  PVC  End 
Caps 

2 

2 

12”  PVC  pipe 

4 

0 

8”  PVC  pipe 

2 

4 

4  y2  ”  PVC  pipe 

2 

8 

Sandpaper 

1 

0 

Tape  measure 

1 

0 

CQD  Forms 


CQ  DUTY  REPORT 

SUBJECT  NUMBER: 

== _ 

LOOT, ON 

PERIOD  COVERED 

FRQM  l  T0 

You  should  do  the  following: 

•  Radio  SPC  Smith  at  the  Guard  Shack  and  report 

that  you  are  taking  over  CQ  Duty 

•  Assemble  a  PVC  footstool  for  CPT  James. 

•  File  the  CQ  Duty  Report  in  folder  marked 

“Completed  CQ  Duty  Reports”. 

•  Return  all  supplies  and  materials  to  their  original 

locations  before  completing  the  exercise. 

•  Place  the  footstool  on  the  CQ  desk  at  the  end  of 

the  exercise. 

•  Radio  SPC  Smith  at  the  Guard  Shack  when 

you’ve  completed  the  exercise. 

Exactly  5  minutes  after  you  start  the  exercise: 

•  Radio  the  1 SG  and  report  the  number  of  vacant 

barracks. 

•  Ask  about  any  additional  tasks  you’re  to  complete  on 

your  shift. 

RULES 

•  You  should  carry  out  all  of  these  tasks 

but  may  do  so  in  any  order. 

•  Assemble  the  footrest  only  in  the 

Assembly  Area. 

•  Bring  only  the  number  of  PVC  parts 

needed  for  the  footrest  from  the  Supply 

Closet  to  the  Assembly  Area. 

•  Do  not  remove  any  signs  or 

instructions  from  the  walls  in  the  work 

areas. 

•  Figure  out  how  to  complete  the 
exercise  in  7  transits  or  less.  In  order 
to  score  the  most  points,  your  trips 
between  zones  should  be  kept  to  a 
minimum. 

•  Take  as  little  time  to  complete  this 
exercise  as  possible  without 
excessively  rushing. 

•  Do  not  ask  questions  for  further 
guidance  about  this  exercise  once  the 
test  starts. 

You  should  obtain  the  following  information  during  the 
exercise: 

Write  the  information  here. 

How  many  3-way  PVC  connectors  are  left  in  stock  after  the 
footrest  has  been  assembled? 

What  is  the  mailing  address  for  Formufit  Inc.  (manufacturer  of 
footstool  parts)? 

What  is  the  telephone  number  of  the  Service  member  who  is 
scheduled  for  CQ  Duty  next  Wednesday? 

What  room  number  is  PVT  Sullivan  in? 

Subject  # 


Inventory  Form 

Item 

Current  Stock 

Needed 

3  way  Elbow  PVC 
Connectors 

T-  PVC 

Connectors 

90  degree  Elbow 
Connectors 

External  PVC  End 
Caps 

12”  PVC  pipe 

8”  PVC  pipe 

4  !4  ”  PVC  pipe 

Sandpaper 

Tape  measure 

CQD  Signs  and  Handouts 


Communications  Roster 


Radio  Holders 

Radio  Frequency 

Military  Police 

7 

Guard  Shack 

10 

Staff  Duty  Officer 

2 

Troop  Medical  Clinic 

4 

Emergency  Room 

9 

Commander 

6 

1SG 

5 

Chaplain 

1 

Orderly  Room 

8 

FULLY  STOCKED  FOOTSTOOL 
PARTS  INVENTORY 


Quantity 

Parts 

8 

1”  diameter  3-Way  PVC  Connectors 

7 

1”  diameter  T-  PVC  Connectors 

6 

1”  diameter  90  Degree  Elbow  PVC 

Connectors 

4 

1”  diameter  External  PVC  End  Caps 

4 

1”x  12”  PVC  pipe 

6 

1”  x  8”  PVC  pipe 

10 

1  ”  x  4  14”  PVC  pipe 

1 

Sandpaper 

1 

Tape  measure 

Contact  List  for  CQ  Duty 


Name 

Contact  Number 

SGT  Michaels 

212-756-4594 

SPC  Anderson 

703/555-5564 

SPC  Jones 

812/464-9804 

SGT  Zavala 

812/484-9493 

PFC  Davis 

410/776-2762 

SGT  Rains 

812/278-9473 

SPC  Jacobs 

561/957-4899 

HOW  TO  USE  THE  WALKIE-TALKIE* 


1. 

2. 
3. 


Push  down  the  MODE  button  and  hold  it  down  to  TURN  ON. 
Use  the  CHANNEL  BUTTON  to  select  the  radio  frequency. 
Hold  down  the  SIDE  BUTTON  and  talk  into  the  walkie  talkie. 


Relevant  only  for  pictured  version  of  Cobra  MicroTalk. 


CQ  DUTY  SCHEDULE 


THIS  WEEK 

NEXT  WEEK 

MON 

TUES 

WED 

THUR 

FRI 

SAT 

SUN 

MON 

TUES 

WED 

THUR 

FRI 

SAT 

SUN 

SGT  Michaels 

X 

X 

SPC  Anderson 

X 

X 

SPC Jones 

X 

X 

SGT  Zavala 

X 

X 

PFC  Davis 

X 

X 

SGT  Rains 

X 

X 

SPC  Jacobs 

X 

X 

BARRACKS  LAYOUT 


RM  301 

PVT  Manning 

RM  302 

Vacant 

RM  303 

RM  304 

SPC  Bridge 

Vacant 

RM  305 

RM  306 

PFC  Lowery 

PVT  Peterson 

RM  307 

RM  308 

PFC  Cooper 

PVT  Sullivan 

RM  309 

RM  310 

Vacant 

SPC  Mahoney 

RM  311 

RM  312 

Vacant 

PFC  Zavela 

RM  101 

PVT  SMITH 

RM  102 

PFC  Wright 

RM  103 

RM  104 

SPC  Moore 

SPC  Munroe 

RM  105 

RM  106 

Vacant 

PVT  Daniels 

RM  107 

RM  108 

PVT  Hernandez 

SPC  Belanger 

RM  109 

RM  110 

PVT  Thomas 

Vacant 

RM  111 

RM  112 

SPC  Saunders 

PFC  Jones 

RM  201 

SPC  Mitchell 

RM  202 

Vacant 

RM  203 

RM  204 

PFC  Jefferies 

SPC  Alexander 

RM  205 

RM  206 

PFC  DeLeon 

Vacant 

RM  207 

RM  208 

Vacant 

PFC  Jefferies 

RM  209 

RM  210 

SPC  Carlisle 

SPC  Donohue 

RM  211 

RM  212 

PFC  Jacobs 

PVT  James 

PVC  FOOTREST  PARTS  AND  ASSEMBLY 


ASSEMBLY  AREA 


SUPPLY  CLOSET 


BULLETIN  BOARD 


CQ  DESK 


RUN-ROLL-AIM  (RRA) 


Run-Roll- Aim  (RRA) 

Task  Description  and  Set  Up 


I.  Description:  The  SM  completes  a  high  level  mobility  task  with  multiple  maneuvers  while 
carrying  a  simulated  weapon.  Maneuvers  are  cued  by  a  computer  screen  with  a  handheld  remote 
controlled  slide  advancer.  The  task  requires  a  rapid  start,  avoiding  a  “trip  wire”  obstacle, 
performing  a  3-5  second  rush,  combat  rolling,  searching  for  visual  targets  through  simulated 
weapon  scope,  rapid  lateral  dodging  and  back  pedaling. 

II.  Purpose:  This  task  requires  the  SM  to  demonstrate  high  level  balance  and  mobility  skills  not 
unlike  those  required  in  a  battlefield  situation,  alternating  between  quick  position  changes  and 
focused  visual  search  through  a  weapon  scope.  Rapid  head  and  body  position  changes  stimulate 
the  vestibular  system,  so  SMs  with  vestibular  impairment  may  have  particular  difficulty  with  this 
task. 


III.  mTBI-related  task  challenges:  Primary  •  Secondary  o 


Cognitive 

Sensorimotor 

Physical 

Executive 

function 

Memory 

Attention 

Reaction 

time 

Eye  gaze 
tracking 

Scanning 

Vestibular 

Balance 

Bend  - 
lift 

Exertion 

Manual 

UE 

Speed 

O 

O 

• 

• 

• 

O 

IV.  Source:  This  task  was  created  by  AMMP  team  members  as  a  way  to  challenge  high  level 
mobility  skills  in  a  situation  simulating  combat. 

V.  Materials  and  Supplies 

•  Clipboard  and  Score  sheet 

•  Stopwatch 

•  Simulated  weapon 

•  Scope  designed  for  bird/insect  viewing  mounted  to  weapon 

•  Adjustable  headband  and  waistband  to  mount  inertial  sensors  (strapped  onto  each) 

•  2  NexGen  inertial  sensors*  and  wireless  access  point  for  data  collection  with  computer 
laptop 

•  Power  point  presentation  of  targets  and  cues 

•  Remote  to  advance  Powerpoint  visual  cues/targets  during  task 

•  5’xlO’  floor  mat  for  landing  in  after  3-5  second  rush,  visual  scanning  component,  and 
combat  rolls  (mat  should  be  secured  as  necessary  to  the  floor  so  that  it  doesn’t  move 
easily) 

•  2,  12”  cones  to  set  up  “trip  wire”  obstacle  (obstacle  created  by  taut  cord  stretched 
between  the  two  cones) 

•  Taped  stripe  down  the  middle  of  the  floor  mat  to  indicate  landing  zone  after  3-5  second 
rush  and  taped  X  on  right  and  left  (316  feet  from  either  side  of  the  center)  to  mark 
approximate  end  point  for  combat  rolls 


*I2M  Sensors  can  be  found  at  this  website  http://www.nexgenergo.com/ergonomics/I2M- 
IMUs.html 

Inertial  sensors  are  placed  (1)  on  an  adjustable  headband  slightly  to  the  opposite  side  of 
the  forehead  from  the  subject’s  eye  used  for  sighting  through  the  weapon  scope  and  (2) 
on  an  adjustable  waist  band  fitted  tightly  around  the  subject  with  the  sensor  in  the  mid 
lumbar  area. 

VI.  Test  Task  Set  Up 

Space  estimate:  Wide  hallway  or  treatment  room  with  space  that  will  accommodate  a  course 
including  minimum  10’  width  and  45’  length  (allow  a  ~30’  run  to  floor  mat  with  stimulus  laptop 
positioned  — 12’  away  from  front  edge  of  mat,  with  short  distance  for  deceleration  at  start/finish). 
Width  of  area  for  trip  wire  placement  and  combat  rolls  on  mat  is  10’  at  a  minimum.  See  Figures 
1-3  for  further  specification. 

Note:  Positioning  of  the  computer  and  mat  must  be  checked  to  assure  that  it  is  possible  to  clearly 
view  the  numbers  on  the  computer  screen  through  the  scope  that  has  a  limited  focal  range,  but 
not  be  viewable  with  the  naked  eye.  It  may  be  necessary  to  modify  the  size  of  the  visual  stimuli 
on  the  computer  screen  if  there  is  insufficient  space  to  allow  the  computer  to  be  placed  12’  away 
from  the  floor  mat. 


Figure  1.  Layout  of  Run-Roll-Aim  task 


Start  and  End 


Figure  2.  Initiation  of  Run-Roll-Aim  task  -  a  taut  cord  is  stretched  between  two  medium 
sized  cones  (12”]  in  height,  taped  to  each  cone  to  serve  as  a  "trip  wire”. 


Figure  3.  Service  member  during  visual  search  after  combat  roll  to  the  right.  Note  she  is 
positioned  on  the  right  X  after  the  combat  roll.  Tape  stripe  down  the  middle  of  the  mat 
indicates  target  for  initial  prone  landing  after  3-5  second  rush. 


Run- 

Roll- 


Run-Roll- Aim  (RRA) 

Examiner  Instructions  and  Script 

This  task  is  cued  by  a  series  of  PowerPoint  slides  advanced  by  a  slide  advancing  remote.  The 
slide  deck  includes  slides  you  can  use  to  illustrate  the  task  as  you  are  providing  instructions 
during  the  walk-through,  followed  by  two  practice  trials.  Use  one  practice  trial  with  everyone, 
use  the  second  practice  trial  only  if  the  subject  has  difficulty  in  smoothly  completing  the  task  on 
the  first  practice  trial  (e.g.  requires  cues,  gets  confused  about  the  sequence,  and  makes  errors 
in  the  task).  Before  administering  the  test,  practice  using  the  slide  set  and  advancing  each 
slide  at  the  appropriate  time  during  the  task.  The  slide  for  direction  of  the  roll  should  be 
advanced  as  the  subject  is  moving  into  the  prone  position  on  the  mat.  Other  cueing  slides  can 
be  advanced  as  the  subject  is  doing  the  prior  component  of  the  movement  sequence. 

INTRODUCTION 

This  test  is  called  Run  Roll  Aim.  It  looks  at  your  agility,  speed,  and  ability  to  find  visual 
targets  during  simulated  military  maneuvers.  It  is  important  to  be  quick  but  also  to  be 
accurate  in  identifying  visual  targets. 

INSTRUCTIONS 

A1 .  Walk-Through  the  Task 

•  Each  part  of  the  task  will  be  cued  by  slides  on  the  computer  at  the  end  of  the  course, 
to  keep  you  moving  through  the  task.  Point  to  computer. 

•  Notice  how  the  course  is  laid  out  in  a  T.  You  will  begin  at  this  end  line  facing  the 
center  of  the  T.  Point  to  the  mat  positioned  in  front  of  computer. 

•  At  the  start  of  each  trial,  you  will  see  this  “READY?”  slide.  Once  you  confirm  that 
you  are  ready,  then  I  will  say  “Set,  Go”.  At  the  same  time,  the  screen  will  also 
indicate  “GO”. 

•  Let’s  walk  through  the  parts  of  the  task  so  you  get  familiar  with  them. 

•  Move  quickly  stepping  over  this  trip  wire  and  perform  a  3-5  second  rush,  landing 
prone  with  your  chest  on  the  center  of  the  mat  on  the  floor  (as  indicated  by  the  tape). 
Have  the  subject  get  into  the  prone  on  the  mat. 

•  Each  time  you  do  the  task,  you  will  see  a  letter  indicating  which  direction  you  should 
ROLL.  If  you  see  an  “L”,  you  should  roll  to  your  left.  If  you  see  an  “R”,  you  should 
roll  to  your  right. 

•  Let’s  say  you  see  an  “R”  on  the  screen.  Do  a  combat  roll  to  your  right.  You  should 
end  with  your  chest  on  the  “X”  marked  on  the  mat  (as  indicated  by  tape). 

•  After  you  roll,  10  numbers  will  appear  on  the  screen.  There  are  5  odd  and  5  even 
numbers  on  each  screen.  Before  each  trial  I  will  tell  you  whether  you  will  be  looking 
for  ODD  or  EVEN  numbers. 

•  Look  through  the  scope  now  to  focus  it  so  that  you  can  see  the  numbers  clearly. 
Have  participant  read  5  even  or  odd  target  numbers  to  ensure  scope  is  properly  focused. 

•  During  each  trial,  you  will  say  the  target  numbers  out  loud  so  that  I  can  hear  you. 
Keep  track  of  the  numbers  so  you  say  each  number  ONLY  ONCE  and  you  know 
when  you  get  all  5  numbers. 

•  Once  you  say  all  5  numbers,  stand  and  SIDE  SHUFFLE  to  the  other  side  of  the  mat 
and  land  in  prone  again  on  the  X  marked  on  the  opposite  side  of  the  mat.  It  is 


important  that  you  position  yourself  on  the  X  so  that  the  scope  will  be  focused 
correctly. 

•  You  will  see  a  new  screen  of  numbers.  If  you  looked  for  ODD  numbers  on  the  first 
side,  you  will  look  for  ODD  numbers  on  the  other  side.  Call  out  all  5  target  numbers 
again.  Have  the  participant  call  out  target  numbers  again. 

•  Once  you  say  all  5  numbers,  ROLL  back  to  the  center. 

•  Stand  and  BACKPEDAL  to  the  starting  point  as  quickly  as  you  can,  but  avoid  the  trip 
wire  on  the  way  back. 

•  Do  you  have  any  questions? 

A2.  Practice  Trial 

Now  let’s  do  a  practice  trial  so  you  can  put  all  the  parts  of  the  task  together. 

Have  the  subject  perform  a  practice  trial. 

Respond  to  questions,  and  provide  cues  as  needed. 

•  As  you  get  familiar  with  the  sequence,  you  do  not  have  to  wait  for  the  next  screen  to 
cue  you.  If  you  get  mixed  up  during  the  task  or  forget  something,  I  may  give  you  a 
cue  so  you  do  not  get  stuck.  In  general  just  do  the  best  you  can,  going  as  quickly 
and  accurately  as  possible. 

CUEING 

Cueing  during  the  practice  trial(s)  and  before  the  first  trial: 

•  This  time  you’re  looking  for _ (even  or  odd)  numbers. 

•  Remember  that  the  letter  on  the  screen  tells  you  which  direction  to  roll.  Provide  this 
cue  for  both  practice  trials  and  the  first  test  trial  OR  in  response  to  an  incongruent  trial  (i.e. 
the  subject  rolled  in  the  wrong  direction). 

•  Ready,  Set,  Go. 

If  during  the  practice  trial  the  subject  does  the  entire  movement  sequence  WITHOUT 
HESITATION  or  ERROR,  move  on  to  testing.  If  the  subject  is  confused  or  requires  cues, 
conduct  a  second  practice  trial.  Record  performance  of  these  trials  on  the  score  sheet.  If  the 
subject  does  not  appear  to  be  clear  on  what  is  an  odd  or  even  number,  complete  both  practice 
trials. 

Cueing  for  second  through  fourth  trials  (if  previous  trial  was  without  errors  in  roll  directions): 

•  This  time  you’re  looking  for _ (even  or  odd)  numbers. 

•  Ready,  Set,  Go. 

Cues  during  task  when  subjects  have  difficulty: 

In  general,  provide  cues  if  the  subject  needs  some  input  to  continue  and  complete  the  task 
successfully.  If  the  subject  identifies  4  numbers  and  then  hesitates,  so  that  you  can  provide  a 
cue  for  “one  more”,  do  so  to  prompt  the  subject  to  search  for  the  fifth  number.  However,  if  the 
subject  identifies  4  numbers  and  begins  to  side  shuffle  or  roll,  allow  the  error  to  occur  without  a 
cue.  (I.e.  Do  not  cue  the  subject  in  the  middle  of  the  side  shuffle  or  roll  so  that  they  must  return 
to  target  position.) 


If  the  subject  does  the  first  visual  search  and  identifies  ALL  incorrect  numbers  (i.e.  identifies  all 
ODD  numbers  when  the  target  was  EVEN  numbers),  provide  a  cue  before  the  next  visual 
search  by  saying  “Your  target  is _ (even  or  odd)  numbers”. 

If  the  subject  asks  if  their  target  is  even  or  odd  after  the  trial  has  begun,  provide  guidance,  but 
mark  as  requiring  a  cue. 

During  visual  search,  if  number  identification  includes  a  delay  greater  than  a  few  seconds  (this 
sometimes  happens  after  3  or  4  targets  have  been  identified),  cue  the  number  of  additional 
numbers  that  are  missing.  If  a  cue  for  “one  more”  results  in  the  subject  identifying  an  incorrect 
number  or  one  that  has  already  been  identified,  provide  an  additional  cue  by  saying  “OK”  and 
progress  to  the  next  segment  of  the  task.  Do  not  insist  on  the  number  identified  being  correct. 

Note  all  errors  in  the  error  column.  Errors  include  skipped  numbers,  incorrect  (even  or  odd) 
numbers  that  are  identified,  and  numbers  that  are  not  included  on  the  slide  at  all.  The  latter 
error  of  commission  sometimes  occurs  when  the  subject  has  a  visual  acuity  problem. 
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Study  ID: 

Rater: 

Date/Time: 

Order: 

RUN-ROLL-AIM  TEST 
Score  Sheet 

Did  the  subject  complete  the  task?  □  Yes  □  No  If  No:  Efexaminer  stopped  Subject 
stopped 

Reason: _ 

Test  session  start  time: _  Dominant  eye _ Left  _ Right 


Practice  trial  -  R  congruent/even  Ti 

ime  (min:sec:100,h  sec): 

Correct  responses 

Errors* 

Cues* 

a.  Avoids  obstacle 
(trip  wire) 

fwd:  □  bkwd:D 

fwd:  □  bkwd:D 

b.Stroop  [mark 
responses] 

rolls  rD 

hesitate  Dself-correct  D 
rolls  L  □ 

c.  Visual  target  ID 

A 

14  20  38  46  52 
□  □  □  □  □ 

19  27  31  45  53 
□  □  □  □  □ 

Other  errors:  D  D  D 

□  □  □ 

d.  Visual  target  ID 

B 

10  68  76  82  94 
□  □  □  □  □ 

17  63  71  89  95 
□  □  □  □  □ 

Other  errors:  D  D  D 

□  □  □ 

e.  Follows  other 
instructions:  side 
shuffle,  backpedal 

All  correct: 

□ 

□  □  □ 

(e.g.  rolls  when  shouldn’t) 

□  □  □ 

f.  Total  trial  scores 

f.1. Total  boxes  marked 
above  /1 4 

f.2.Total  boxes  marked 
above: 

f.3.Total  boxes 
marked  above: 

If  requires  more  than  one  cue  during  practice  trial  1 ,  include  second  practice. 


Practice  trial  -  L  congruent/odd  Time  (min:sec:1 00th  sec): 

Correct  responses 

Errors* 

Cues* 

a.  Avoids  obstacle 
(trip  wire) 

fwd:  □  bkwd:D 

fwd:  □  bkwd:D 

b.Stroop  [mark 
responses] 

rolls  L  □ 

hesitate  Dself-correct  D 
rolls  R  □ 

c.  Visual  target  ID 

A 

19  27  31  45  53 
□  □  □  □  □ 

14  20  38  46  52 
□  □  □  □  □ 

Other  errors:  D  D  D 

□  □  □ 

d.  Visual  target  ID 

B 

17  63  71  89  95 
□  □  □  □  □ 

10  68  76  82  94 
□  □  □  □  □ 

Other  errors:  D  D  D 

□  □  □ 

e.  Follows  other 
instructions:  side 
shuffle,  backpedal 

All  correct: 

□ 

□  □  □ 

(e.g.  rolls  when  shouldn’t) 

□  □  □ 

f.  Total  trial  scores 

f.1. Total  boxes  marked 
above  /1 4 

f.2.Total  boxes  marked 
above: 

f.3.Total  boxes 
marked  above: 

Observations/Comments: 


Study  ID: 

Rater: 

Date/Time: 

Order: 

Trial  1  -  L  incongruent/odd  1 

fime  (min:sec:100,h  sec): 

Correct  responses 

Errors* 

Cues* 

a.  Avoids  obstacle 
(trip  wire) 

fwd:  G  bkwd:EH 

fwd:  G  bkwd:G 

b.Stroop  [mark 
responses] 

rolls  L  G 

hesitate Gself-correct  G 
rolls  R  G 

c.  Visual  target  ID  C 

29  37  43  51  65 
□  □  □  □  □ 

24  30  48  56  62 
□  □□□□ 

Other  errors:  G  G  G 

□  □□ 

d.  Visual  target  ID  D 

13  29  75  81  97 
□  □  □  □  □ 

12  28  74  86  90 
□  □□□□ 

Other  errors:  G  G  G 

□  □□ 

e.  Follows  other 
instructions:  side 
shuffle,  backpedal 

All  correct: 

□ 

□  □□ 

□  □□ 

f.  Total  trial  scores 

f.1. Total  boxes  marked 
above  /1 4 

f.2.Total  boxes  marked 
above: 

f.3.Total  boxes 
marked  above: 

Observations/comments: 


Trial  2  -  R  congruent/even  Time  (min:sec:100,h  sec): 

Correct  responses 

Errors* 

Cues* 

a.  Avoids  obstacle 
(trip  wire) 

fwd:  G  bkwd:G 

fwd:  G  bkwd:G 

b.Stroop  [mark 
responses] 

rolls  RG 

hesitate  Gself-correct  G 
rolls  L  G 

c.  Visual  target  ID  A 

14  20  38  46  52 
□  □□□□ 

19  27  31  45  53 
□  □□□□ 

Other  errors:  G  G  G 

□  □□ 

d.  Visual  target  ID  B 

10  68  76  82  94 
□  □□□□ 

17  63  71  89  95 
□  □□□□ 

Other  errors:  G  G  G 

□  □□ 

e.  Follows  other 
instructions:  side 
shuffle,  backpedal 

All  correct: 

□ 

□  □□ 

□  □□ 

f.  Total  trial  scores 

f.1. Total  boxes  marked 
above  /1 4 

f.2.Total  boxes  marked 
above: 

f.3.Total  boxes 
marked  above: 

Observations/comments: 


Study  ID: 

Rater: 

Date/Time: 

Order: 

Trial  3  -  L  congruent/even  Time  (min :sec:1 00th  sec): 

Correct  responses 

Errors* 

Cues* 

a.  Avoids  obstacle 
(trip  wire) 

fwd:  EH  bkwd:EH 

fwd:  G  bkwd:G 

b.Stroop  [mark 
responses] 

rolls  L  G 

hesitate Gself-correct  G 
rolls  R  G 

c. Visual  target  ID  B 

10  68  76  82  94 
□  □  □  □  □ 

17  63  71  89  95 
□  □□□□ 

Other  errors:  G  G  G 

□  □  □ 

d.Visual  target  ID  C 

24  30  48  56  62 
□  □  □  □  □ 

29  37  43  51  65 
□  □□□□ 

Other  errors:  G  G  G 

□  □  □ 

e.  Follows  other 
instructions:  side 
shuffle,  backpedal 

All  correct: 

□ 

□  □□ 

□  □  □ 

f.  Total  trial  scores 

f.1  .Total  boxes  marked 
above  /1 4 

f.2.Total  boxes  marked 
above: 

f.3.Total  boxes 
marked  above: 

Observations/comments: 


Trial  4  -  R  incongruent/odd  Ti 

ime  (min:sec:100,h  sec): 

Correct  responses 

Errors* 

Cues* 

a.  Avoids  obstacle 
(trip  wire) 

fwd:  G  bkwd:G 

fwd:  G  bkwd:G 

b.Stroop  [mark 
responses] 

rolls  RG 

hesitate  Gself-correct  G 
rolls  L  G 

c.Visual  target  ID  A 

19  27  31  45  53 
□  □□□□ 

14  20  38  46  52 
□  □□□□ 

Other  errors:  G  G  G 

□  □□ 

d.Visual  target  ID  D 

13  29  75  81  97 
□  □□□□ 

12  28  74  86  90 
□  □□□□ 

Other  errors:  G  G  G 

□  □  □ 

e.  Follows  other 
instructions:  side 
shuffle,  backpedal 

All  correct: 

□ 

□  □□ 

□  □  □ 

f.  Total  trial  scores 

f.1. Total  boxes  marked 
above  /1 4 

f.2.Total  boxes  marked 
above: 

f.3.Total  boxes 
marked  above: 

Observations/comments: 


‘Scoring  guidance: 

Errors  =  contact  with  trip  wire,  incorrect/repeated/skipped/commission  number(s)  recited,  hesitation,  self-correction 
and/or  wrong  direction  roll  in  Stroop,  roll  instead  of  side  shuffle,  out  of  sequence  movement(s)  during  task 
Cues  =provided  DURING  trial  for  completion  of  task  (e.g.  “one  more”  number,  confirmation  of  odd/even,  instruction 
about  side  shuffle  vs.  roll,  etc.) 

Each  ROW  must  have  at  least  one  box  (or  more)  marked.  Tally  number  of  X  in  each  column  in  row  f. 

Reinstruction  BETWEEN  trials  is  not  marked  as  an  error  OR  cue. 


Run-Roll-Aim 
SCORING  GUIDE 


Examiner  scoring  supplies/materials: 

•  Stopwatch 

•  Remote  to  operate  stimulus  slides  on  computer 

•  Clipboard 

•  Pencil 

•  Score  sheet 

Definitions  of  key  underlying  concepts: 

Time  -  time  to  complete  the  Run  Roll  Aim  trial  from  GO  signal  to  when  the  first  foot 
crosses  the  finish  line. 

#  Items  correct  -  number  of  visual  targets  correctly  identified  during  each  trial 

#  Items  in  error  -  errors  made  in  obstacle  avoidance,  Stroop  response,  visual  target 
identification  (odd  when  even  targets,  e.g.),  repeated  correct  items  in  a  set,  lack  of 
following  of  task  instructions.  Specific  criteria  during  the  components  of  the  trial  are 
described  below. 

Cue  -  examiner  prompts  or  instruction  reminders  provided  during  the  trial  (e.g.,  “one 
more”  number,  confirmation  of  odd/even  before  number  identification  begins,  cue  about 
odd/even  in  between  number  identification  sets  when  first  set  was  wrong  target, 
instruction  about  side  shuffle  vs.  roll  etc.) 

Scoring  procedures  for  performance  subscores 

Before  starting  the  task 

Ask  service  member  which  eye  he/she  uses  to  aim  with  weapon  site.  Place  inertial 
sensor  olose-just  off  center  on  the  opposite  side  to  avoid  it  getting  in  the  way  during  the 
task. 

If  using  inertial  sensor(s)  for  data  collection,  record  the  start  time  to  allow  match  to 
sensor  time  codes  in  recorded  trials. 

Fill  out  the  following: 

-  Subject’s  study  ID 

-  Your  Rater  ID 

-  Today’s  date 

-  Where  in  the  test  order  the  subject  is  performing  this  test-task 

-  Inertial  sensor  location(s),  if  applicable 

Scoring  for  each  trial  uses  the  same  procedure,  so  the  specifics  will  only  be  described 
once.  All  that  varies  is  the  direction  of  the  roll  that  is  cued,  whether  they  are  to  call  out 
odd  or  even  numbers  and  if  there  is  a  congruent  or  incongruent  Stroop  cue. 

Before  each  trial,  circle  the  word  even  or  odd  as  a  cue  to  yourself  to  inform  the 
participant  which  they  are  to  call  out  during  the  trial.  Reinstruction  between  trials  is  not 
marked  as  an  error  or  cue.  Also,  see  Scoring  Guidance  box  at  end  of  score  sheet. 


During  each  trial: 


Performance 

dimension 


Scoring  procedures 


Performance 

subscore 


Time 


Hand  time  each  trial,  starting  the  stopwatch  when  you  advance  the 
slide  to  GO  and  say  GO  at  the  same  time.  End  timing  when  the 
participant’s  first  foot  crosses  the  end  line. 

In  the  event  that  there  is  a  mistrial,  and  a  condition  must  be 
repeated,  record  time  codes  for  each  trial  to  indicate  which  are  to  be 
retained  (from  inertial  sensor  trials  on  computer). _ 


N/A  -  no 
subscore 


a.  Avoided 
obstacles 


Error  =  contact  with  trip  wire 

Check  “fwd”  and  bkwd”  boxes  in  either  “Correct  responses”  or 
“Errors”  column  based  on  whether  or  not  SM  had  contact  with  trip 
wire  during  run  toward  or  away  from  the  mat. 


N/A  -  no 
subscore. 


Correct  responses 

Errors* 

a.  Avoids  obslacte 
(trip  wire) 

fwd.  0  ftkwd:Q 

fwd:  D  bkwd:DT 

b.  Stroop 
response 


Error  =  hesitation,  self-correction,  or  wrong  direction  rolled 

Indicate  whether  or  not  the  SM  rolled  in  the  correct  direction  by 
checking  the  box  in  the  “Correct  Response”  column  or  checking  1  of 
the  3  error  boxes  in  the  “Errors”  column. 


N/A  -  no 
subscore 


b.Stroop  [mark 

rolls  L  DT 

responses] 

hesilateO  self-correct  CH 
rolls  R  □ 


Hesitation  occurs  when  SM  takes  1  second  or  longer  to  initiate  the 
roll.  Completion  of  a  roll  or  any  part  of  it  to  the  wrong  direction  is 
checked.  In  this  example,  the  SM  rolled  to  the  R  partly,  but  then 


corrected  and  rolled 

1  - r  _ 

L;  three  boxes  are  chec 

;ked 

b.Sttoop  [mark 
:  responses] 

rolls  L  V 

hesitate  —self-correct  Cd 
rolls  R  OK 

c.  &  d.  Visual 
targets  ID  [X] 


Errors  =  incorrect,  repeated,  skipped  numbers,  or  reporting  a 
number  that  is  not  on  the  screen  (commission) 

Check  the  boxes  corresponding  to  numbers  on  the  computer  screen 
that  the  SM  calls  out.  Some  or  all  will  be  in  the  “Correct  responses” 
column  or  “Errors”  column,  which  should  include  reporting  numbers 
other  than  those  presented  on  the  screen. 

In  the  “cues”  column,  check  a  box  for  each  time  a  cue  was  offered 
regarding  visual  target  (e.g.,“one  more”  -  indicating  there  is  one 
more  number  that  needs  to  be  recited  before  they  reach  the  target 
ones). 


N/A  -  no 
subscore 


d.  Visual  target  ID  Q 


12  28  74  66  90 

□□□□□  ^ 
Other errors:  D  Dll1 


If  the  SM  recites  odd  when  even  is  the  target,  provide  a  cue  that  odd 
is  the  target  before  they  do  the  second  set,  marking  cue  in  the  third 
column  for  the  next  set. 

e.  Follows 
other 

instructions: 
side  shuffle, 
back  pedal 

Errors  =  running  forward  versus  back  pedaling  as  instructed;  side 
shuffling  versus  rolling  as  instructed 

If  the  SM  follows  instructions  to  side  shuffle  and  back  pedal,  check 
“All  correct”  box  in  the  “Correct  responses”  column. 

For  each  error  made  (e.g.,  runs  forward  back  to  finish  line) 

Check  a  box  in  the  “Errors”  column  each  time  the  SM  does  not 
perform  as  instructed  (e.g.  runs  forward  back  to  finish  line,  does  not 
correct  roll  made  in  wrong  direction,  rolls  when  he  should  side 
shuffle). 

N/A  -  no 
subscore 

Observations- 

comments 

N/A  -  no 
subscore 

At  task  completion 


Performance 

dimension 

Scoring  procedures 

Performance  subscore 

f.  Total  trial  scores 

Count  the  number  of  boxes  checked  in 
the  “Correct  responses"  column  for  rows 
a,  b,  c,  d,  e  (of  14  possible) 

f.1  =  total  number  of  boxes 
checked  in  the  “Correct 
responses”  column 

Count  the  number  of  boxes  checked  in 
the  “Errors”  column  for  rows  a,  b,  c,  d,  e 

f.2  =  total  number  of  boxes 
checked  in  the  “Errors”  column 

Count  the  number  of  boxes  checked  in 
the  “Cues”  column  for  rows  c,  d,  e 

f.3  =  total  number  of  boxes 
checked  in  the  “Cues”  column 

RRA  Scoring  Frequently  Asked  Questions 

Avoids  obstacle 

What  if  the  subject’s  foot  lightly  touches  the  trip  wire  during  the  forward  rush? 
This  constitutes  an  error. 

Correct  scoring: 


Correct  responses 

Errors* 

a.  Avoids  obstacle  (trip  wire) 

fwd:  □  bkwd:EH 

fwd:  ^  bkwdO 

Stroop  responses 

Are  there  trials  in  which  SM  are  more  prone  to  rolling  in  the  wrong  direction? 
Hesitation  is  more  likely  for  trial  1  and  4  with  incongruent  cues.  However,  count  any 
hesitation  as  an  error. 


How  will  I  know  if  I  see  a  hesitation  ? 

A  hesitation  occurs  when  the  subject  is  viewing  the  directional  screen  and  waits  for 
more  than  1  second  before  initiating  the  roll  to  either  direction. 


Identifies  visual  target 

How  should  I  score  if  the  subject  asks  if  the  target  is  even  or  odd  after  the  trial  has  begun? 

Provide  guidance,  but  mark  as  requiring  a  cue. 

How  would  I  score  errors  and  cues  provided  in  the  following  situations? 

The  subject  identified  4  numbers  and  then  hesitates.  I  provided  a  cue  for  “one  more” 
and  the  subject  then  reported  the  5th  number. 

Correct  scoring: 


Correct  responses 

Errors 

Cues 

C.  Visual  target  ID  B 

10  68  76  82  94 

[3  [3  M  EH  EH 

17  63  71  89  95 

□  □  □  □  □ 
Other  errors:  □  □  □ 

El  □□ 

The  subject  identifies  4  numbers  and  begins  roll  (in  the  correct  direction).  I  did  not 
provide  a  cue  because  it  was  in  the  middle  of  the  roll. 


Correct  scoring: 


Correct  responses 

Errors 

Cues 

b.  Stroop  [mark 
responses] 

rolls  L  ^ 

hesitate  EHself-correct  EH 
rolls  R  EH 

c.  Visual  target  ID  B 

10  68  76  82  94 

EH  EH  EH  EH  □ 

17  63  71  89  95 
□  □  □  □  □ 

Other  errors:  EH  EH  EH 

□  □  □ 

Follows  other  instructions:  side  shuffle,  back  pedal 

How  should  I  score  it  if  the  SM  rolls  after  the  first  visual  target  identification  set  instead 
of  standing  to  side  shuffle? 

If  the  SM  completes  the  roll  quickly  (doesn’t  appear  to  realize  the  error),  let  the  trial 
continue  and  advance  to  the  next  visual  target  ID  slide.  After  that  trial,  review  the 
sequence  and  encourage  him/her  to  watch  the  slides  for  the  side  shuffle  cue  on  the  next 
trial.  Mark  the  error  and  don’t  mark  “all  correct”.  If  the  SM  hesitates  for  more  than  one 
second  or  begins  to  roll  and  it’s  possible  to  provide  a  cue  to  correct  it,  cue  about 
standing  to  side  shuffle,  and  mark  a  cue  on  the  scoresheet.  If  they  complete  the 
stand/side  shuffle,  mark  all  correct. 

Correct  scoring: 


Correct  responses 

Errors 

Cues 

e.  Follows  other 
instructions:  side  shufft 
backpedal 

All  correct:  EH 

□  □  □ 

IEI  □  □ 

RRA  PowerPoint  Slides 


READY? 


L  or  R 


14 


45 


22 


50 
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46 


53 


27 


31 


38 


SIDE  SHUFFLE  TO 
OPPOSITE  SIDE, 
TO  PRONE 


14 


45 


22 


50 


19 


46 


53 


27 


31 


38 


BACK  PEDAL  TO 

START 


Practice  Trial  1 


READY? 


14 


45 


22 


50 


19 


46 


53 


27 


31 


38 


SIDE  SHUFFLE  TO 
OPPOSITE  SIDE, 
TO  PRONE 


63 


10 


89 


17 


82 


94 


68 


95 


76 


71 


ROLL  to  CENTER 


BACK  PEDAL  TO 

START 


Practice  Trial  2 


READY? 


14 


45 


22 


50 


19 


46 


53 


27 


31 


38 


SIDE  SHUFFLE  TO 
OPPOSITE  SIDE, 
TO  PRONE 


63 


10 


89 


17 


83 


94 


68 


95 


76 


71 


ROLL  to  CENTER 


BACK  PEDAL  TO 

START 


L  incongruent  -0 


READY? 


24 


SI 


48 


62 


37 


29 


65 


56 


43 


30 


SIDE  SHUFFLE  TO 
OPPOSITE  SIDE, 
TO  PRONE 


12 


23 


75 


28 


86 


74- 


97 


30 


13 


ROLL  to  CENTER 


BACK  PEDAL  TO 

START 


R  congruent  E 


READY? 


14 


52 


27 


46 


31 


SIDE  SHUFFLE  TO 
OPPOSITE  SIDE, 
TO  PRONE 


63 


10 


89 


17 


82 


94 


68 


95 


76 


71 


ROLL  to  CENTER 


BACK  PEDAL  TO 

START 


L  congruent  - E 


READY? 


63 


10 


89 


17 


82 


94 


68 


95 


76 


71 


SIDE  SHUFFLE  TO 
OPPOSITE  SIDE, 
TO  PRONE 


24 


51 


4-B 


G2 


37 


29 


GS 


56 


43 


30 


ROLL  to  CENTER 


BACK  PEDAL  TO 

START 


ROLL  to  CENTER 


R incongruent -0 


READY? 


u 


45 


20 


52 


27 


19 


53 
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31 


3S 


SIDE  SHUFFLE  TO 
OPPOSITE  SIDE, 
TO  PRONE 
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75 
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74 


97 


90 
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ROLL  to  CENTER 


BACK  PEDAL  TO 

START 


PATROL-  EXERTION 


Patrol/Exertion 
Task  Description  and  Set  Up 

I.  Description:  The  SM  is  challenged  to  gather  intelligence  in  a  recorded  video  depicting  a 
virtual  Afghanistan  patrol  environment  while  reporting  observed  EED  markers  based  on  a 
briefing  provided  at  the  beginning  of  the  video.  The  SM  then  uses  the  information  to  answer 
specific  questions  from  memory  at  the  end  of  the  patrol  video.  The  SM  will  perform  continuous 
step-ups  on  an  exercise  step  at  an  intensity  of  65-85%  of  HR  maximum  throughout  the  activity 
while  being  monitored  for  effort  level  via  a  Polar  HR  monitor  and  performance  observation.  The 
SM  will  be  wearing  a  combat  helmet,  eye  protection,  and  be  carrying  a  simulated  M16  weapon 
equipped  with  a  trigger  switch  connected  via  Bluetooth  to  a  computer  configured  to  record 
reaction  time  (RT).  The  SM  is  required  to  press  the  switch  each  time  a  beep  tone  stimulus  is 
heard  throughout  the  video  as  a  measure  of  RT  during  a  divided  attention  multitask. 

II.  Purpose:  Visual  scanning  skills,  attention,  memory,  RT,  and  decision-making  under 
exertional  conditions  are  often  impaired  following  mTBI.  This  task  places  demands  on  divided 
and  alternating  attention,  prospective  memory,  visual  attention  and  scanning,  gaze  stability,  and 
multimodal  (i.e.,  auditory,  visual,  and  vestibular)  processing  in  conjunction  with  simultaneous 
exertional  demands. 


III.  mTBI-related  task  challenges:  Primary  •  Secondary  o 


Cognitive 

Sensorimotor 

] 

Physical 

Executive 

function 

Memory 

Attention 

Reaction 

time 

Eye  gaze 
tracking 

Scanning 

Vestibular 

Balance 

Bend  -  lift 

Exertion 

Manual 

UE 

Speed 

O 

• 

• 

• 

• 

O 

• 

IV.  Source:  This  task  was  created  by  AMMP  team  members  as  a  way  to  challenge  visual  and 
auditory  processing,  reaction  time,  cognitive  processing  and  attention  skills  in  a  situation 
involving  moderate  and  simultaneous  exertional  demand. 

V.  Materials  and  Supplies 

Materials: 

•  4-5”  high  exercise  step, 

•  heart  rate  monitor  (sports  type  monitor  with  a  chest  strap  and  wrist  watch  component  to 
allow  examiner  to  monitor  exercise  heart  rate), 

•  Table  with  elevated  platform  for  positioning  video  monitor  at  eye  level  when  subject  is 
stepping 

•  laptop  with  Patrol  video  and  RT  programs 

•  video  monitor  with  external  speakers 

•  recording  device  for  recording  SM  responses  to  post  Patrol  questions, 

•  standard  helmet  and  eye  protection, 

•  Simulated  M-16  rubber  weapon  configured  with  RT  switch, 

•  Antifog  wipes  for  eye  protection, 

•  Disinfecting  wipes  for  cleaning  HR  monitor,  helmet,  eye  protection  and  weapon  between 
subjects 


VI.  Test  Task  Set  Up 

Space  estimate:  Approximately  6x8  foot  area 


Patrol  Task  -  Exertion  with  Reaction  Time 
Examiner  Instructions  and  Script 

Open  the  Patrol  Task  application  on  the  laptop,  make  a  profile  for  subject.  Use  the 
subject  ID  number  and  click  “create  profile.”  Turn  on  the  switch  on  the  blue  gun. 

INTRODUCTION 

This  task  involves  repeated  stepping  onto  this  exercise  step  while  you  watch  a 
virtual  patrolling  video  set  in  a  rural  countryside.  We  will  monitor  your  heart  rate 
in  order  to  keep  you  exercising  at  a  moderate  pace.  How  old  you  are  so  we  can 
calculate  your  exercise  heart  rate? 

Write  age  and  calculate  APMHR. 

INSTRUCTIONS 

A.  Don  Equipment 

•  Please  put  this  heart  rate  monitor  on  under  your  shirt  at  the  breastbone. 
You  will  need  to  wet  this  part  a  little. 

Hand  subject  the  sports  HR  monitor  and  point  to  the  contact  area  on  the  underside  of 
the  POLAR  label,  that  they  should  get  a  little  wet,  and  turn  the  wrist  watch  monitor  to 
EXE  (note  it  takes  a  10-15  seconds  to  start  indicting  HR). 

You  have  to  stay  within  5  feet  of  the  subject  with  the  watch  part;  it  works  to  clip  in  on  to 
the  top  of  your  clipboard. 

•  You  will  wear  a  helmet,  eye  protection,  and  carry  this  blue  weapon. 

Hand  all  equipment  to  subject.  Allow  the  subject  to  choose  the  size  helmet  he/she 
wants  to  wear.  If  the  subject  is  wearing  glasses,  additional  eye  pro  is  not  necessary. 

B.  Symptom  Assessment 

Before  we  start,  please  rate  two  things  for  me. 

Point  to  RPE  scale. 

•  Using  this  chart  from  6  (which  means  no  feeling  of  exertion)  to  20  (very,  very 
hard  exertion),  rate  how  hard  you  are  working  while  standing  still. 

Point  to  vision  chart. 

•  Using  this  chart  where  0  means  “you  see  clearly  or  normally”  and  10  means 
“the  worst  or  most  unstable  my  vision  could  be,”  rate  your  vision  as  it  is 
now. 

I  am  going  to  ask  you  to  rate  those  two  items  again  for  me  while  you  are 
stepping  up  and  down. 

C.  Practice  Reaction  Time 

•  During  this  task  we  will  test  your  reaction  time.  When  you  hear  this  tone 


Trigger  the  sound  of  the  stimulus  using  the  computer  mouse  several  times — click  on 
the  BEEP  icon. 

•  Press  this  small  switch  as  quickly  as  you  can. 

Point  to  switch  on  blue  rubber  weapon,  just  below  trigger. 

•  Let’s  practice  this  a  couple  of  times. 

•  Each  time  you  hear  the  beep,  Trigger  stimulus  on  computer  by  pushing  the 
“beep”  icon,  push  the  switch  as  fast  as  you  can.  Allow  subject  to  practice 
pushing  the  switch  to  manual  stimulus  several  times. 

•  Do  you  have  any  questions? 

D.  Test  Reaction  Time  in  standing 

•  Now  let’s  record  your  reaction  time.  The  computer  will  randomly  trigger  the 
“BEEP”  sound  a  few  times  in  the  next  30  seconds. 

•  READY  to  push  that  switch  as  fast  as  you  can? 

Start  the  RX  time  trial.  The  computer  will  play  2  beep  sounds  at  2  random  times  in 
30  seconds;  it  will  be  at  least  8  seconds  for  the  first  tone  sound. 

Record  RT  (in  msec)  from  the  computer  screen  on  to  score  sheet  (la  and  1b). 

•  If  the  numbers  are  greater  than  400,  try  one  additional  trial  and  say  “Let’s 
practice  once  more...  Push  the  switch  as  fast  as  you  can  after  you  hear  the 
sound.” 

•  Record  the  2nd  trial  below  the  first  on  the  score  sheet. 

E.  Tactical  pauses  and  Reporting 

•  Now  I  will  describe  the  video  to  you.  The  video  will  provide  instructions  and 
a  review  of  common  IED  MARKERS  that  you  should  be  looking  for  during 
each  of  4  tactical  pauses. 

•  Each  pause  begins  when  the  patrol  leader  says  “initiating  tactical  pause”. 
The  video  will  show  a  virtual  360  degree  turn.  You  will  continue  stepping 
during  the  pause.  During  each  turn,  call  out  ANY  IED  MARKERS  you  see 
based  on  the  instructions  you  were  given.  The  tactical  pause  will  end  when 
the  patrol  leader  says  “OK,  tactical  pause  complete,  let’s  keep  moving” 

•  At  the  end  of  the  video,  I  will  hand  you  a  recorder  and  ask  questions  that 
may  include  intelligence  concerning  your  unit’s  location,  relevant  times  and 
date  within  the  virtual  scenario,  as  well  as  details  about  individuals, 
equipment,  or  activities  you  observe  while  on  the  video  patrol. 

Show  subject  the  hand  held  recorder. 


•  The  details  for  the  final  questions  must  be  answered  from  memory  so  pay 
close  attention  to  important  information  during  the  video  for  this  final 
report. 

F.  Rules  and  Brief-Back 

•  You  must  follow  these  rules: 

•  Push  the  switch  every  time  you  hear  the  beep. 

•  Call  out  the  IED  markers  ONLY  during  the  tactical  pauses  where  the 
patrol  leader  says  “Initiating  Tactical  Pause”  to  start,  and  “Tactical 
Pause  COMPLETE”  to  end. 

•  Save  all  other  tactical  observations  for  the  questions  at  the  end. 

•  Step  continuously  throughout  the  video  until  I  tell  you  to  stop. 

•  What  are  the  4  tasks  you  will  do  during  this  task? 

Subject  should  say: 

1)  Press  switch  in  reaction  to  beep, 

2)  Identify  IED  markers  observed  during  tactical  pauses, 

3)  Answer  questions  at  the  end  of  the  video 

4)  Step  continuously. 

•  Do  you  have  any  questions?  I  want  you  to  know  that  you  won’t  be 
shooting  at  anyone  and  no  one  will  shoot  at  you,  and  nothing  blows  up 
during  this  video.  It  is  important  for  you  to  react  as  fast  as  you  can  and  to 
report  everything  as  well  as  you  can.  DO  YOUR  BEST!! 

G.  Reaction  Time  during  stepping 

•  Now  we  will  test  your  reaction  time  with  the  beeps  agatn-when  you  begin 
stepping. 

•  Ready?  Begin  stepping 

•  Start  the  reaction  time  trial  again. 

•  Record  RT  (in  msec)  from  the  computer  screen  onto  score  sheet  (1c  and  Id). 

B  (repeat).  Symptom  Assessment  white  stepping 

Point  to  RPE  scale. 

•  Using  this  chart  again  from  6  to  20,  rate  how  hard  you  are  working  now. 

Point  to  vision  chart. 

•  Using  this  chart  from  0  to  10,  rate  your  vision  now. 


I’m  going  to  start  the  video  now.  Keep  stepping  until  I  tell  you  to  stop.  The 
beep  will  sound  multiple  times  during  the  video.  Make  sure  you  respond 
quickly  every  time  you  hear  it. 


DURING  THE  TEST 

Monitor  HR  throughout  to  keep  in  range  which  you  take  off  the  chart  (APMHR). 

Goal  is  65%  of  APMHR  (age  predicted  maximum  heart  rate)  by  the  time  the  Courtyard 
patrol  scene  starts  so  if  he/she  isn’t  close  after  the  first  2  minutes  of  instructions  on 
video,  say  Step  a  little  faster  when  nothing  important  is  showing  on  the  video.  Rarely  if 
the  subject  is  over  85%  of  APMHR,  you  may  have  to  say  “slow  down  a  bit.”  Record 
the  HR  in  the  HR  column  next  to  each  tactical  pause,  as  well  as  below  the  RPE  score  at 
the  bottom  of  the  score  sheet. 

Turn  your  recorder  on  at  about  the  “SWITCH  TO  YOUR  NODS”  statement  on  the  video. 

END  OF  THE  TEST 

B  (repeat)  Symptom  Assessment:  Ask  the  Workload  and  Gaze  stability  questions  again 
right  at  the  end  of  the  video  while  subject  is  stepping.  Point  to  the  wall  charts  when  you 
ask  these  questions.  Say: 

•  Keep  stepping  and  rate  how  hard  you  are  working. 

•  How  stable  is  your  vision? 

Have  subject  stop  stepping  and  ask  post  video  questions  (see  score  sheet).  Press  the 
“record”  button. 

State:  “Recording  Patrol  Report  with  DRAGON  XXX  NOW”  (where  XXX  is  the 

subjects  ID  number)  and  then  hand  recorder  to  subject.  Press  the  “STOP”  button  on 
the  recorder  when  done  with  post  patrol  questions. 

EXAMINER  GUIDANCE: 

IMMEDIATELY  AFTER  THE  TEST: 

•  Record  the  msec  of  the  1 1  reaction  times  at  the  end  of  the  score  sheets.  NOTE  that  if  they  do 
not  react  or  if  it  takes  them  longer  than  2  seconds,  then  the  screen  will  have  “-1”  which  should  be 
recorded  in  the  appropriate  blank-, 

•  Turn  the  speaker  switch  off  on  the  BLUE  WEAPON  at  the  end  of  the  task. 

•  The  battery  should  be  changed  when  the  battery  indicator  on  the  screen  states  75%  or  less. 

•  Clean  the  helmet,  eye  pro,  HR  monitor  and  strap,  and  blue  weapon  and  use  the  antifog  wipes  on 
the  eye  pro  after  each  subject. 

OTHER  ADMINISTRATION  TIPS 

•  If  the  eye  pro  fogs  up,  have  the  subject  take  them  off  during  the  PATROL  Task. 

•  Close  and  reopen  the  PATROL  Application  between  subjects.  If  it  locks  up,  close  and  open  the 
application  again  and  re-do  profile.  Open  the  PATROL  Application  prior  to  turning  on  the  switch 
on  the  blue  weapon. 

•  There  are  separate  instructions  for  the  instrumented  blue  weapon  software. 


Study  ID: 


Rater: 


Date: 


Order  #: 


SM  Completed  Task?  _ Yes _ No  (examiner  stopped) 


_No  (subject  stopped) 


AGE 


PATROL  TASK  with  exertion  Score  Sheet 
220-  Age  = _ APMHR  65% _ APMHR  85% 


Initial  Reaction  time  (la) _ ms  (1b) _ ms  (stand)(1c) _ ms  (Id) _ ms 

(stepping) 

NOTES: _ 


RATER:  Place  check  mark  in  box  to  indicate  correct  response  (No  negative  for  extra 
observations) 


Heart 

Rate 

#1 

OBSERVATION  /  IED  Marker 

IDENTIFIED 
(1  Point  Ea) 

FOOT  BRIDGE 
(NEAR  SIDE) 

Rock  linefar  side  of  creek  that  point  down  along 
the  river  road,  (at  30°) 

Trash  Pile  #1  (across  bridge  at  0°) 

Conspicuous  Box  (at  270°) 

Overturned  earth  (at  270°) 

TOTAL  #  CORRECT 

A.  /4 

#2 

OBSERVATION  /  IED  Marker 

IDENTIFIED 
(1  Point  Ea) 

GROVE 

ENTRANCE 

3  parallel  line  marks  low  on  the  wall  (at  30°) 

(Also:  “chalk  marks”) 

Red  prayer  flag  (under  rock  inside  wall-at 

330°)(Also:  red  cloth,  flag,  material...,  red  rug; 
“prayer...”  or  “red....”) 

TOTAL  #  CORRECT 

B.  12 

#3 

OBSERVATION  /  IED  Marker 

IDENTIFIED 
(1  Point  Ea) 

MID  GROVE 

Broken  tree  branch  (at  90°) 

Small  line  of  rocks  just  below  broken  branch  (at 

90°) 

Small  pile/bundle  of  sticks  (also:  logs,  twigs, 
branches(at  270°) 

TOTAL  #  CORRECT 

C.  13 

#4 

OBSERVATION  /  IED  Marker 

IDENTIFIED 
(1  Point  Ea) 

END  GROVE 

Stick  bundle  stacked  vertically  on  R  side  of  gate 
(at  30°) 

3  parallel  line  marks  low  on  the  wall  (at  330°);  or 
“chalk  marks” 

Small  line  of  rocks  (at  330°) 

Overturned  earth  (at  0  or  360°  depends  on  when 
they  see  it) 

TOTAL  #  CORRECT 

D.  /4 

SCANNING  SUB-SCORETOTAL  #  CORRECT  (A+B+C+D) 

X.=  /1 3 

COMMENTS  (extra  items/objects  reported)/ SYMPTOMS; 

Post  Patrol  Questions 

Correct  answer  (Examples) 

#  Correct 

1  pt  each  unless  indicated 

What  enemy  vehicles  did  you  see? 

Motorcycle,  cycle,  moped,  bike,  MC,  Harley,  any 
word  that  indicates  motorcycle  (1) 

/I 

What  were  the  last  grid  coordinates 
reported? 

EB  2682  (2  pt  for  all  correct)  1  pt  each  for  letters  or 
numbers  correct,  must  be  exact  (0,  1 ,  or  2) 

12 

What  color  clothing  were  the  kids  in 
the  grove  wearing? 

White  (1) 

/I 

What  surveillance  equipment  did  you 
see? 

“binoculars”,  “scope”,  “binos”,  “observation  device” 

(1) 

/I 

What  weapon  did  the  individual  on 
the  motorcycle  have? 

Knife,  blade,  sword,  dagger,  janbiya  or  khanjar  (1) 
Accept  any  term  that  indicates  they  identify  a  type 
of  knife 

/I 

What  time  did  the  patrol  enter  the 
grove? 

Tolerances:  1700-1705  hours  (1) 

/I 

What  was  the  date  that  this  activity 
occurred? 

Tol:  20  June  2013  (month/day  only  is  acceptable) 

(D 

/I 

What  items  did  you  see  throughout 
the  scenario  that  could  be  used  to 
create,  arm  or  detonate  an  IED? 

Tolerances:  “Jugs,  containers,  lEDs,  gas  cans, 
water  cans,  red  caps,  fuel  tanks,  yellow  jugs,  daisy 
chain”  (1) 

/I 

Tolerances:  “Wire”,  “cord”,  “fuse”,  “Det  cord”  (1) 

/I 

Tolerances:  “Battery”,  “9  volt”,  “power  cell”  (1) 

/I 

PATROL  QUESTIONS  SUBSCORE 

Y.=  /II 

Anything  else  that  you  noticed  that  1 
should  include  in  my  report? 

(Write  comment(s)  free  form) 

SCANNING  SUBSCORE  (A  +  B  +  C  + 
D)=X 

Copy  from  bottom  of  table  front  side 

X.=  /1 3 

PATROL  SUB-SCORE 

Copy  from  above 

Y.=  /II 

TOTAL  POINTS  (SCANNING 
+SALUTE)  (  X  +  Y) 

Total 

CNJ 

II 

isi 

Reaction  time  (2)  ms  (3)  ms  (4)  ms  (5)  ms  (6)  ms  (7)  ms 

(8)  ms  (9)  ms  (10)  ms  (11)  ms  (12)  ms 

Patrol-Exertion 
Scoring  Guidelines 


Examiner  scoring  supplies/materials: 

•  Clipboard 

•  Pencil 

•  Subject  score  sheets  and  administration  instructions 

•  Wall  signs  for  RPE  [rate  of  perceived  exertion]  and  Vision  Clarity  [Likert  Scales] 

•  Hand  held  voice  recorder,  heart  rate  monitor,  Instrumented  blue  mock  M-16  weapon 
with  Reaction  time  software/hardware,  helmet,  eye  protection. 

•  Computer  and  Monitor  for  playing  PATROL  video  and  to  run  reaction  time  program. 

•  Patrol  video  cued  up,  turn  on  speakers,  turn  on  switch  on  blue  weapon  [for  Rx  time 
component] 

Before  starting  the  task,  the  rater  fills  out  the  following: 

o  Study  ID,  rater,  date  and  test  order  [1st,  2nd,  ...6th  of  the  test  tasks] 

o  Age 

o  Calculate  the  age  predicted  maximum  heart  rate 

o  Determine  65%  and  85%  range  of  APMHR  for  exercising — calculate  or  use  the  chart 

o  Enter  the  subject  ID  number  into  the  computer  program  on  laptop  and  click  "Create 
Profile"  [This  is  the  program  for  running  the  baseline  Reaction  Time  software. 

1]  In  the  left  hand  column  of  the  score  sheet  under  "Heart  Rate",  indicate  the 
approximate  time  into  the  video  when  cues  are  given  to  the  subject  to  "speed  up”  or 
to  "slow  down”  in  order  to  keep  the  subject’s  exercise  HR  in  the  65-85%  APMHR 
range  per  protocol. 

2]  Reaction  time — Record  the  initial  reaction  time  while  standing  [2a  and  2b]  and 
while  stepping  [2c  and  2d].  Read  off  computer  screen  after  initial  trial  and  write  the 
milliseconds  down  in  the  appropriate  spaces. 

3]  While  standing,  while  initially  stepping  before  the  video  starts  and  at  the  end  of 
stepping  before  the  post-video  questions,  record  the  subject  reported  number  for 
rate  of  perceived  exertion  [RPE]  from  the  6-20  and  vision  clarity  from  0-10  in  the 
appropriate  box  on  the  score  sheet.  Record  any  other  comments  or  reported 
symptoms  in  the  appropriate  blank  space  below  the  RPE  and  vision  clarity 
questions. 

4]  Tactical  pause  1-4  [AKA  SPOT  reports]  — make  a  check  mark  in  each  box  that  the 
subject  correctly  identifies  and  write  down  in  the  blank  space  any  extra  words  or 
comments  that  subjects  makes  during  the  tactical  pause  or  while  patrolling. 

5]  Add  up  points  for  each  tactical  pause  in  blanks  A  through  D  and  record  the  total  [out 
of  13]  in  box  X;  also  record  on  the  back  side  of  the  score  sheet  in  box  X. 


6)  Mark  the  box  for  each  component  of  the  Post  Patrol  Questions  using  a  1  or  2  as 
appropriate  per  the  examples. 

7)  Add  up  each  section  of  the  Post  Patrol  Questions  and  record  total  under  Y  [out  of  11 
maximum  points)  and  also  copy  the  score  below  in  the  subscore  section  Y. 

8)  Add  up  the  subscore  summary  boxes  for  the  Scanning  [SPOT  Reports)  -  X,  Post 
Patrol  Questions-  Y  and  fill  in  the  TOTAL  in  Box  Z. 

9)  Copy  the  Reaction  time  numbers  [in  msec)  off  the  computer  screen  into  Reaction 
time  blanks  2-12  on  the  bottom  of  the  page. 


Patrol  Scoring  Guidelines/Tolerances 

1.  Tactical  Pause:  Any  "items  of  interest”  reported  on  the  IED  marker  list  will  be  credited 
regardless  of  when  they  are  identified  [during  the  tactical  pause  or  after).  In  general,  the 
participant  should  provide  the  examiner  feedback  during  or  immediately  after  the  tactical 
pause  so  be  prepared. 

2.  Erroneous  identifications  [markers,  motorcycle  track,  etc.)  and  IED  component  materials 
[battery,  jugs,  and  detonation  cord)  offered  during  the  tactical  pause  [e.g.  craters,  out  of 
place  dirt  mounds,  etc.)  will  not  be  counted  as  errors  of  commission  [no  points  deducted). 

3.  Description  of  rock  lines  or  stick  piles  must  denote  deliberate  placement  by  enemy  forces 
[e.g.  rock  line  or  stick  pile  deliberately  placed,  rock  cairn,  etc.)  not  just  "rocks  and 
branches” 

4.  During  1st  Tactical  pause,  subject  should  identify  the  box  on  the  near  side  of  the  river  [not 
the  brick  on  the  far  side). 

5.  Post  Patrol  Questions — see  the  middle  column  on  the  scoresheet  labelled  "Correct  answer 
[Examples)”  for  expected  answers.  Write  any  additional  comments  in  the  blank  spaces.  No 
penalty  is  given  for  additional  words  or  answers;  they  are  just  recorded  in  score  sheet 
blanks. 


Patrol-  Exertion 
Materials 


BORG  RPE  Scale 


6  NO  EXERTION  AT  ALL 

7 

EXTREMELY  LIGHT 

8 

9  VERY  LIGHT 

10 

11  LIGHT 

12 

13  SOMEWHAT  HARD 

14 

15  HARD  (HEAVY) 

16 

17  VERY  HARD 

18 

19  EXTREMELY  HARD 

20  MAXIMAL  EXERTION 


VISION 


0123456789  10 


Normal, 
Clear,  & 
Stable 
Vision 


Extremely 
Blurry  or 
Jumpy 
Vision 
"The 
Worst  It 
could  be" 


System  Description 

The  PATROL  Reaction  Test  measures  reaction  time  events  during  a  video  of  a  patrol 
mission  scenario.  When  the  pre-set  time  event  is  reached  in  the  video,  reaction  from  the 
subject  is  prompted  by  an  audible  cue  from  the  Trigger  module.  The  Subject  is  asked  to 
react  to  the  audible  cue  by  pressing  a  button  located  on  the  pistol  grip.  When  the  button  is 
pressed,  the  reaction  time  is  measured  as  the  delay  from  the  audible  cue  to  when  the 
button  is  pressed.  The  reaction  time  is  calculated  and  logged  by  the  PC  software.  Hardware 
as  installed  on  rubber  duck  can  be  seen  in  Figure  1. 


Figure  1  -  Trigger  installed  on  rubber  duck 


Trigger  Module 

The  trigger  module  is  mounted  at  the  bottom  of  the  magazine  on  the  blue  rubber  duck,  as 
seen  in  Figure  2  and  consists  of  a  bluegiga  blel!2  Bluetooth  Low  Energy  module,  a  buzzer 


Figure  2  -  Trigger  module  mounted  on  rubber  duck 


Electronics  Design 

The  electronics  are  all  wired  and  soldered  to  the  blell2  module  according  to  the 
schematics  in  Figure  3. _ 


3V 


TITLE:  trigger  module 

Document  Number: 

REU: 

□ate:  7/23/2015  11:53:58  AM  Sheet:  1/1 


Figure  3  -  Electrical  schematic 


Appendix  A  -  Installation  Instructions 
Installation  of  software  on  PC 

Make  sure  the  latest  java  version  is  installed  on  the  PC.  The  PC  software  in  its  existing 
folder  structure  should  be  copied  to  a  folder  on  the  PC,  for  example  "C:\Patrol”.  Copy 
rxtxSerial.dll  to  the  JAVE_JRE/bin  folder.  Run  the  software  through  PATROL.jar. 


User  Instructions  for  PATROL  Test 

The  system  consists  of  the  Trigger  Module  (located  on  the  blue  gun),  the  Dongle  (mounted 
in  a  USB  port)  and  the  PATROL  Software. 

Setup  Instructions 

Preparing  the  system  for  a  test  involves  the  following  steps: 

1)  Start  the  PC  running  the  PATROL  Software 

2)  Make  sure  that  the  Dongle  is  mounted  in  the  correct  USB  port  (the  front  most  port  on 
the  right  hand  side  of  the  PC).  Any  USB  port  used  must  be  2.0  compatible. 

3)  Turn  on  the  Trigger  Module  and  look  for  the  green  light  to  come  on 

a)  If  the  green  light  does  not  come  on,  please  read  the  Troubleshooting  guide  on  how 
to  replace  the  battery 

4)  Start  the  PATROL  Software  located  on  the  Desktop  of  the  PC 

5)  The  first  dialog  that  comes  up  prompts  you  to  select  the  COM-port  that  the  Dongle  is 
connected  to,  press  OK 

6)  The  Software  will  now  try  to  connect  to  the  Trigger  Module,  which  will  result  in  a  beep 
from  the  Trigger  Module. 

a)  If  the  beep  is  not  heard  and  an  error  dialog  pops  up,  please  read  the 
Troubleshooting  guide  on  how  to  connect  the  Trigger  Module 

Operating  Instructions 

As  the  system  is  set  up  and  the  Trigger  Module  is  on  with  a  green  light  and  the  PC  running 
the  PATROL  Software.  Follow  these  steps  to  run  the  test: 

1)  Use  the  Beep  button  to  test  the  beep  sound  and  the  connection  to  the  Trigger  Module 

2)  Create  a  new  profile  by  entering  the  Profile  ID  in  the  corresponding  text  box 

3)  Press  Create  Profile 

4)  Run  Reaction  Test  and  Run  PATROL  Test  are  now  available 

5)  Press  Run  Reaction  Test  to  run  the  reaction  test,  the  results  will  populate  the  Results 
text  box  when  the  test  has  finished 

6)  Press  Run  PATROL  Test  to  run  the  video  and  the  automated  test  sequence 

a)  At  any  time  during  the  PATROL  Test,  the  test  can  be  paused  by  pressing  SPACE 

b)  At  any  time  during  the  PATROL  Test,  the  test  can  be  aborted  by  pressing  ESC 

7)  After  the  finished  test,  the  results  will  populate  the  Results  text  box 

a)  Write  down  the  results 

b)  A  backup  of  the  test  results  will  also  be  saved  to  the  hard  drive  of  the  PC 

8)  When  done  testing,  TURN  OFF  the  Trigger  Module  to  conserve  battery  and  exit  the 
Software 


Troubleshooting 

Replacing  the  Trigger  Module  battery 

The  battery  in  the  Trigger  Module  is  a  3V  CR2032  with  about  200mAh  capacity.  The  battery 
should  last  for  about  10  hours  of  testing,  please  follow  these  steps  to  replace  a  depleted 
battery: 

1)  Turn  the  Trigger  Module  off 

2)  Open  the  bottom  casing  with  a  flat  object  by  inserting  it  in  the  slot  on  the  opposite  side 
of  the  green  light. 

3)  Twist  the  flat  object  so  that  the  casing  pops  off 

4)  Use  the  flat  object  to  pry  the  battery  out  of  its  holder 

5)  Insert  the  new  battery  with  the  positive  facing  towards  you 

6)  Snap  the  lid  back  on  to  the  Trigger  Module 

7)  Power  on  and  confirm  that  the  green  light  comes  on 

a]  If  the  green  light  does  not  come  on  after  replacing  the  battery,  please  contact 
support 

Re-connecting  the  Trigger  Module 

At  times,  the  PATROL  Software  will  not  recognize  the  Trigger  Module.  The  following  can  be 
reasons  for  the  Trigger  Module  not  being  recognized: 

•  The  Trigger  Module  is  not  started  before  the  Software  is  started 

•  A  new  Trigger  Module  is  used 

•  Interference  during  the  connection  phase 

•  Hardware  malfunction,  please  contact  support 

The  following  steps  will  describe  how  to  connect  the  Trigger  Module  at  the  event  of  a 
connection  failure: 

1)  Make  sure  the  Trigger  Module  is  on,  the  green  light  must  be  on 

2)  Open  the  configuration  dialog 

3)  In  the  configuration  dialog,  press  Discover 

4)  The  list  above  the  Discover  button  will  be  populated  with  available  devices  to  connect 
to 

5)  Select  the  "PATROL  Trigger"  and  press  Connect,  this  should  result  in  a  beep  from  the 
Trigger  Module 

6)  Save  and  Close  to  store  the  new  settings 

7)  The  configuration  dialog  will  exit  and  a  beep  from  the  Trigger  Module  will  confirm  the 
connection. 

8)  If  this  does  not  work,  please  contact  support 


Rx  Time  Project  AMMP  Conceptual  Design 

Reaction  time  measuring  during  the  PATROL  scenario.  The  subject  will  be  exposed  to  11 
auditory  cues  throughout  the  scenario  and  will  respond  by  pressing  a  trigger  in  proximity 
to  a  "blue  gun".  The  system  will  consist  of  software  running  on  a  PC  and  a  separate  Trigger 
Module  that  can  output  auditory  cues  as  well  as  measure  the  delay  between  the  cues  and 
trigger  reactions  from  the  subject.  The  interface  between  the  PC  and  the  Trigger  Module 
will  be  wireless  to  allow  flexible  mounting  as  well  as  not  to  interfere  with  the  subject's 
movement.  The  Trigger  module  must  on  its  own  measure  the  time,  since  a  PC  cannot  be 
considered  a  reliable  real-time  system,  this  is  especially  true  considering  communication 
with  an  external  trigger  button. 

PC  Software 

The  software  will  encapsulate  the  video  showing  the  PATROL  scenario.  Researcher  can 
configure  software  to  set  the  time  of  the  triggers.  Configuration  should  be  done  in  a 
configuration  file.  The  software  will  track  video  frames  and  transmit  a  command  to  the 
Trigger  Module  when  a  pre-selected  frame  is  reached. 

Trigger  Module 

Trigger  module  will  receive  a  command  from  the  PC  software  to  start  an  auditory  cue.  The 
on-board  microcontroller  will  start  a  timer  when  the  cue  is  started.  The  subject  will  press  a 
trigger  that  will  stop  the  timer  and  the  time  interval  is  calculated  by  the  microcontroller. 
The  time  interval  is  transmitted  to  the  PC  software.  The  PC  software  presents  reaction 
times  at  the  end  of  the  video. 

Hardware 

A  PC  running  the  software,  wireless  USB-dongle  (or  built-in  Bluetooth),  wireless  module, 
microcontroller  board,  digital  trigger  button,  piezo  speaker  (or  similar),  battery,  battery 
charging  circuit  (or  replaceable  battery)  and  LED. 

Specific  Example 

The  PC  software  could  be  developed  in  C#/.NET  which  with  existing  libraries  would  make 
this  process  quick.  The  PC  could  have  built-in  Bluetooth  capabilities  or  a  separate  dongle  is 
used.  The  Trigger  module  would  be  a  custom  design  optimized  for  size  and  weight.  A 
lower-range  MSP430  16-bit  microcontroller  would  be  mounted  on  a  custom  PCB  together 
with  a  SMD  trigger  button,  a  Bluegiga  Bluetooth  RN-42  module,  a  status  LED,  a  power 
button  and  a  pizeo  speaker.  Contingent  on  calculations  for  power  needs,  the  battery  would 
probably  be  a  lithium  3V  coin  cell  battery.  A  custom  plastic  enclosure  will  only  expose  the 
power  switch,  status  LED  and  trigger  button  to  the  user. 

These  recommendation  are  mostly  based  on  my  experience  with  the  aforementioned 
technologies,  depending  on  the  developer's  experience,  other  environments  might  be  more 
familiar  and  preferred. 


Contact 

Patrol  Trigger  Module 

Daniel  Nilsson 

dasnilsson@gmail.com 

612-702-2919 

Line  # 

Qty 

Ref 

Manufacturer/Distributer 

Manufacturer  Part# 

Description 

1 

1 

Ul 

bluegiga 

BLE112-A-vl 

Bluegiga  BLE  112-A  BLE  module 

2 

1 

SP1 

Mu  rata 

PKM22EPPH4001-B0 

Piezo  4kHz  Buzzer 

3 

1 

S2 

Sparkfun(dist) 

COM-09190 

12  mm  Momentary  pushbutton  switch 

4 

1 

SI 

Sparkfun(dist) 

COM-00102 

SPDT  Mini  power  switch 

5 

1 

BAT1 

Renata 

SMTU2032-LF 

CR2032  Battery  Holder 

6 

1 

LED1 

Sparkfun(dist) 

COM-09592 

Green  5mm  LED 

7 

1 

R1 

Yageo 

MFR-25FRF52-150R 

Resistor  150  ohm 

8 

1 

Cl 

Panasonic 

EEU-FC0J101B 

Aluminum  Electrolytic  capacitor  100uF/6.3V 

9 

2 

C2,  C3 

TDK 

FK28X5R1C105K 

Ceramic  Capacitor  luF/16V 

ILLINOIS  AGILITY  TEST  (IAT)  -  PACKING  LIST 


Illinois  Agility  Test 
Description  and  Test  Set  up 


I.  Description:  The  Illinois  Agility  Test  requires  running  distances  of  30’  with  rapid  direction 
changes  and  navigation  of  obstacles  in  a  serpentine  pattern  during  the  middle  part  of  the 
obstacle  course.  A  memory  task  is  also  completed.  Then  both  the  agility  task  and  the  memory 
task  are  performed  at  the  same  time.  Accuracy  of  memory  recall  and  time  to  complete  the  agility 
task  are  measured  in  single  and  dual-task  conditions. 

II.  Purpose:  This  task  requires  higher  level  mobility  [rapid  performance  on  an  agility  course) 
while  performing  a  cognitive  task  (7  word  list  memory  task)  at  the  same  time.  This  testing 
protocol  is  similar  to  the  Walking  and  Remembering  Test,  which  has  been  validated  in  both 
older  adults  and  individuals  with  moderate  to  severe  brain  injury. 


II 


.  mTBI-related  task  challenges:  Primary  •  Secondary  o 


Cognitive 

Sensorimotor 

Physical 

Executive 

function 

Memory 

Attention 

Reaction 

time 

Eye  gaze 
tracking 

Scanning 

Vestibular 

Balance 

Bend  - 
lift 

Exertion 

Manual  UE 
Speed 

• 

O 

• 

O 

IV.  Source:  Getchell  B.  Physical  Fitness:  A  Way  of  Life  (2ed).  New  York:  Wiley  and  Sons,  Inc. 
1979. 

V.  Materials  and  Supplies 

•  Colored  masking  tape  to  mark  start  and  end  points  of  agility  course 

•  Clipboard  and  Score  sheet 

•  Stopwatch 

•  6  cones 

•  Adjustable  headband  and  waist  band 

•  NexGen  inertial  sensors*  and  wireless  data  collection  port  and  laptop. 

*I2M  Sensors  can  be  found  at  this  website  http://www.nexgenergo.com/ergonomics/I2M- 
IMUs.html 

Inertial  sensors  are  placed  [1)  on  an  adjustable  headband  slightly  to  the  opposite  side  of  the 
forehead  from  the  subject’s  eye  used  for  sighting  through  the  weapon  scope  and  [2)  on  an 
adjustable  waist  band  fitted  tightly  around  the  subject  with  the  sensor  in  the  mid  lumbar  area. 

VI.  Test  Task  Set  Up 

Space  estimate:  Wide  hallway  space  that  is  40’  long  and  12’  wide  at  a  minimum  to  allow  for 
agility  course  set  up  and  acceleration/deceleration  during  the  agility  task. 

Figures  1  and  2  illustrate  task  set  up. 


Figure  1  Task  set  up  example 


Figure  2  The  starting  position  for  the  task  is  with  the  service  member  prone  with  his/her  hands 
at  the  level  of  the  starting  line.  The  cones  in  the  middle  of  the  course  are  placed  to  allow 
sufficient  room  on  the  ends  to  circle  the  cones  comfortably  (in  this  example  allowing  1.5' 
between  end  cone  and  far  line  and  start/finish  line,  to  avoid  having  to  slow  to  avoid  nearby 
obstacles  just  beyond  the  course). 


STARTING  POSITION 


COURSE  END 


Illinois  Agility  Test 
Examiner  Instructions  and  Script 

Before  testing,  roll  a  die  to  determine  the  order  of  the  word  lists  that  will  be  used  with  the 
subject.  Also  indicate  sequence  of  priority  conditions  based  on  the  subject  ID  number.  Trial  1 
is  single  task  word  list.  Trial  2  is  dual-task  condition  without  instructions.  Trials  3  and  4  are 
priority  conditions.  Odd  ID  number  subjects’  priority  order  is  words  3/agility  4.  Even  ID 
number  subjects’  priority  order  is  agility  3/words  4. 

INTRODUCTION 

This  task  is  called  the  Illinois  Agility  Test.  It  will  assess  your  speed  and  agility  while 
moving  on  an  obstacle  course,  as  well  as  your  ability  to  recall  a  short  list  of  words. 

INSTRUCTIONS 

Single  Task  Condition  -  Walk  Through  Agility  Task 

Show  schematic  of  the  course  (on  page  4  of  this  script). 

•  You  will  begin  here  lying  prone  with  your  hands  at  the  level  of  this  piece  of  tape. 

•  When  I  say  go,  stand  up  and  run  as  quickly  as  you  can  around  the  large  cone  at 
this  end.  Trace  path  on  schematic.  Go  around  it,  without  touching  it,  then  move  to 
the  cones  in  the  middle.  Run  serpentine,  alternating  around  one  side  and  then  the 
other,  through  the  four  cones  in  the  middle  of  the  course  both  UP  and  BACK.  Trace 
path  on  schematic. 

•  Round  the  fourth  cone  in  the  middle  and  then  run  on  the  INSIDE  of  the  large  cone 
on  the  far  right,  before  running  quickly  back  to  the  finish.  Trace  path  on  schematic. 

•  If  you  get  the  sequence  mixed  up,  try  your  best  to  correct  it  by  going  back  to  where 
you  made  the  mistake.  If  you  stop  during  the  trial,  we  will  need  to  repeat  it. 

•  Go  as  quickly  as  you  can,  but  avoid  touching  any  of  the  cones. 

•  If  space  is  limited,  then  add:  Take  care  that  you  don’t  go  so  fast  that  it  is  hard  to  stop 
at  the  end  of  the  course,  since  space  is  limited  in  this  room. 

•  Do  you  have  any  questions? 

A.  Single  Task  Condition  -  Practice  and  Timed  Agility  Task 

•  Now  let’s  have  you  jog  through  the  course  once  to  make  sure  you  have  the 
sequence  right. 

•  Now  let’s  try  a  timed  trial. 

•  Do  you  have  any  questions? 

Record  performance  time.  If  there  are  errors,  the  subject  must  repeat  the  timed  trial  a  second 
time. 

B.  Single  Task  Condition  -  Word  List  Task  /  Cognitive  Task 

Both  the  examiner  and  subject  sit  for  this  part  of  the  task. 

•  Now  I  am  going  to  read  a  list  of  7  words  to  you.  These  are  things  you  might  pack  if 
you  were  going  to  deploy. 


•  Listen  carefully,  because  you  need  to  remember  them  for  a  short  delay  before  you 
repeat  them  back  to  me.  The  delay  is  the  length  of  time  it  took  you  to  complete  the 
agility  task. 

•  I  will  say  the  7  words,  then  I  will  say  “Delay”. 

•  When  I  say  “Now”  tell  me  the  words  you  remember.  You  can  say  the  words  in  any 
order. 

•  Do  you  have  any  questions? 

Answer  all  questions  before  proceeding.  Use  the  number  of  words  remembered  in  the  single 
task  condition  as  the  span  in  the  dual-task  condition,  if  the  number  recalled  is  5  or  greater.  If 
the  subject  recalls  fewer  than  5  words,  use  a  list  of  5  words  in  the  dual-task  condition. 

Read  the  word  list  from  the  score  sheet  that  corresponds  to  the  number  on  the  first  die  that 
you  rolled  before  testing.  Read  words  at  a  rate  of  one  per  second,  dropping  voice  inflection 
slightly  on  the  last  word  in  the  sequence.  When  “Delay”  is  said,  start  the  stopwatch.  Say 
“Now”  when  the  time  for  completion  of  the  agility  task  is  met.  Record  the  order  of  word  recall 
on  the  word  list  sheet  for  those  that  are  correct.  If  a  new  word  is  added  to  the  list,  then  write  it 
down  verbatim  for  that  trial.  Record  errors  by  adding  missed  words  (error  of  omission)  and 
added  words  (error  of  commission)  together. 

C.  Dual-Task  Condition  -  WITHOUT  priority  instructions 

The  subject  can  remain  seated  through  task  instructions. 

•  Now  we  are  going  to  combine  the  agility  task  with  remembering  words.  The  start 
position  will  be  the  same,  in  prone  with  your  hands  at  the  level  of  this  piece  of  tape. 

•  Each  time  we  repeat  the  task  it  will  be  with  a  different  list  of _ (number)  words. 

You  can  forget  the  words  that  you  have  heard  previously.  Just  focus  on 
remembering  the  words  you  have  heard  last.  We  will  do  this  task  a  few  times. 

•  Once  you  have  heard  the  last  word,  there  will  be  a  short  delay  so  you  can  get  the 
last  word  in  your  head.  Then  I  will  say  “Ready,  go”. 

•  Remember  the  words  as  you  run  the  course.  When  you  finish,  tell  me  the  words 
you  remember,  in  any  order. 

•  Complete  the  agility  course  as  quickly  as  you  can,  but  take  care  not  to  touch  any 
cones. 

•  Do  you  have  any  questions? 

With  the  subject  in  the  starting  position,  read  the  word  list  from  the  score  sheet  that 
corresponds  to  the  number  on  the  second  die  that  you  rolled  before  testing.  Read  words  at  a 
rate  of  one  per  second,  dropping  voice  inflection  slightly  on  the  last  word  in  the  sequence. 
Record  the  time  it  takes  the  subject  to  complete  the  course,  and  record  the  words  recalled  for 
each  trial.  Use  a  repeat  trial  if  the  subject  does  not  follow  instructions  or  stops  before 
completing  the  trial. 

If  the  subject  confuses  the  agility  course  sequence  and  does  not  correct  it  (i.e.  the  subject 
does  not  follow  the  serpentine  pattern  in  middle  of  the  course  in  both  directions)  so  that  the 
motor  task  time  is  less  than  the  single  task  time,  then  repeat  the  trial.  If  the  participant 
recognizes  and  corrects  the  error,  then  record  the  time  and  make  a  notation  that  an  error  was 


made  in  the  course  path.  Any  contact  with  cones  during  a  trial  should  also  be  marked  as  an 
error. 

If  errors  are  made  in  word  recall  (i.e.  commissions,  or  partial  recollection  of  compound 
words),  then  mark  those  responses  as  errors.  If  the  subject  misunderstands  how  a  word  is 
pronounced,  then  write  what  he/she  says  verbatim.  If  it  does  not  match  the  word  recited 
exactly,  then  it  is  an  error. 

Offer  a  drink  of  water  or  brief  rest  prior  to  completing  the  remaining  trials.  If,  after  completing 
the  agility  course,  the  subject’s  respiratory  rate  is  increased,  or  if  the  subject  reports 
exertional  symptoms  of  headache  or  dizziness,  then  allow  him  or  her  to  normalize  before  a 
new  trial. 

D.1 .  Word  List  Priority  This  task  is  first  for  subjects  with  odd  ID  numbers. 

•  You  are  going  to  remember  words  and  run  again. 

•  This  time  I  want  you  to  focus  on  remembering  as  many  words  as  you  can  while 
running  the  agility  course. 

•  Let’s  see  if  you  can  remember  all _ (number)  words  this  time. 

•  Are  you  ready? 

Read  the  word  list  from  the  score  sheet  that  corresponds  to  the  number  on  the  die  that  you 
rolled  before  testing.  Read  words  at  a  rate  of  one  per  second,  dropping  voice  inflection 
slightly  on  the  last  word  in  the  sequence.  Record  the  time  it  takes  the  subject  to  complete  the 
course,  and  record  the  words  recalled  for  each  trial.  Use  a  repeat  trial  if  the  subject  does  not 
follow  instructions  or  stops  before  completing  the  trial. 

D.2.  Agility  Priority  This  task  is  first  for  subjects  with  even  ID  numbers. 

•  For  this  trial  of  words  and  running,  I  want  you  to  focus  on  doing  the  agility  task  as 
quickly  as  you  can  while  also  doing  the  memory  task. 

•  Let’s  see  if  you  can  beat  your  fastest  time. 

•  Are  you  ready? 

Read  the  word  list  from  the  score  sheet  that  corresponds  to  the  number  on  the  die  that  you 
rolled  before  testing.  Read  words  at  a  rate  of  one  per  second,  dropping  voice  inflection 
slightly  on  the  last  word  in  the  sequence.  Record  the  time  it  takes  the  subject  to  complete  the 
course,  and  record  the  words  recalled  for  each  trial.  Use  a  repeat  trial  if  the  subject  does  not 
follow  instructions  or  stops  before  completing  the  trial. 
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Study  ID: 

Rater: 

Date/Time: 

Order: 

ILLINOIS  AGILITY  TEST 
Score  Sheet 


Did  the  subject  complete  the  task?  □  Yes  □  No 

If  No:  □  Examiner  stopped  task  □  Subject  stopped  task 

Did  inertial  sensor(s)  malfunction?  □  Yes  □  No 
If  Yes:  □  Head  □  Trunk 

A.  Single  Task  Condition  -  Agility  Testing 

Trial  1 :  Time _ (sec)  Repeat  if  errors  in  path. 

Trial  2:  Time _ (sec) 

B.  Single  Task  Condition  -  Word  List  Task  /  Cognitive  Task 

Complete  this  first  word  list  task  with  a  delay  equivalent  to  the  single  agility  task  time.  Then 
use  the  span  of  words  remembered  in  this  single  word  list  task  in  future  dual-task  conditions. 

Number  of  words  remembered  to  be  used  in  dual-task: _ 

C.  Dual-Task  Condition  -  WITHOUT  priority  instructions 

Mark  correct  words  by  number  (i.e.  1 , 2,  3)  as  they  are  recalled.  Mark  errors  of  omission  from 
a  list  with  an  X.  Record  errors  of  commission  (words  added)  to  the  list  with  an  X.  Words  must 
be  said  verbatim  or  else  it  is  an  error.  Note  any  errors  from  the  agility  course  (i.e.  incorrect 
path,  contact  with  cones). 

D1  &  D2.  Dual-Task  Condition  -  WITH  priority  instructions 
Same  as  C.  above. 


List  1:  Trial#  □  single  □  dual 

List 2:  Trial#  □  single  □  dual 

If  dual:  □  WITH  instr.  □  WITHOUT  instr. 

If  dual:  □  WITH  instr.  □  WITHOUT  instr. 

□  word  priority  □  agility  priority 

□  word  priority  □  agility  priority 

1 .  Rifle 

1.  Helmet 

2.  Camelbak 

2.  Ammo 

3.  Socks 

3.  Tourniquet 

4.  Notebook 

4.  Pen 

5.  Tape 

5.  Eye  pro 

6.  Knee  pads 

6.  Ruck 

7.  Compass 

7.  Chemlight 

Words  Recalled  Correctly: 

Words  Recalled  Correctly: 

Word  Errors: 

Word  Errors: 

Agility  Test  Time: 

Agility  Test  Time: 

Agility  Course  Errors: 

Agility  Course  Errors: 

Study  ID:  Rater: 

Date/Time:  Order: 

List  3  Trial  #  □  single  □  dual 

If  dual:  □  WITH  instr.  □  WITHOUT  instr. 
□  word  priority  □  agility  priority 

List  4  Trial  #  □  single  □  dual 

If  dual:  □  WITH  instr.  □  WITHOUT  instr. 

□  word  priority  □  agility  priority 

1 .  Radio 

1 .  Protractor 

2.  Flashlight 

2.  Knife 

3.  Goggles 

3.  Bandoleer 

4.  Poncho 

4.  Watch 

5.  Magazine 

5.  Jacket 

6.  Bandage 

6.  DEET 

7.  Marker 

7.  Lanyard 

Words  Recalled  Correctly: 

Word  Errors: 

Agility  Test  Time: 

Agility  Course  Errors: 

Words  Recalled  Correctly: 

Word  Errors: 

Agility  Test  Time: 

Agility  Course  Errors: 

List  5  Trial#  □  single  □  dual 

If  dual:  □  WITH  instr.  □  WITHOUT  instr. 
□  word  priority  □  agility  priority 

List  6  Trial  #  □  single  □  dual 

If  dual:  □  WITH  instr.  □  WITHOUT  instr. 

□  word  priority  □  agility  priority 

1 .  E-tool 

1 .  Ear  pro 

2.  Boots 

2.  Batteries 

3.  Pistol 

3.  Sleeping  bag 

4.  Duffel  bag 

4.  Rope 

5.  Canteen 

5.  Cap 

6.  Gloves 

6.  Holster 

7. Iodine 

7.  Scissors 

Words  Recalled  Correctly: 

Word  Errors: 

Agility  Test  Time: 

Agility  Course  Errors: 

Words  Recalled  Correctly: 

Word  Errors: 

Agility  Test  Time: 

Agility  Course  Errors: 
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Illinois  Agility  &  Equipment  List  Dual  Task 
Scoring  Guide 

Examiner  scoring  supplies/materials: 

•  A  die  or  other  means  of  randomly  choosing  word  list  order 

•  Stopwatch 

•  Clipboard 

•  Pencil 

•  Scoresheet 

Definitions  of  key  underlying  concepts: 

Words  Recalled  Correctly  -  the  number  of  word  list  items  that  the  subject  correctly  reports  at 
the  end  of  each  trial 

Word  Errors  -  the  number  of  intrusions  (new  words  added)  or  words  missed  from  the  list 
used  in  a  trial 

Agility  Test  Time  -  time  to  complete  the  agility  course  from  GO  signal  to  when  the  first  foot 
crosses  the  finish  line  (to  the  hundredth  of  a  second). 

Agility  Course  Errors  -  the  number  of  times  within  a  trial  that  the  person  does  not  adhere  to 
course  and/or  requires  cues  to  do  so)  [contacts  a  cone  during  the  course,  misses  the  second 
serpentine  pattern  and  stops  him/herself  to  correct  it,  e.g.] 

Scoring  procedures  for  performance  subscores: 

Before  starting  the  task 
Fill  out  the  following: 

-  Subject’s  study  ID 

-  Your  Rater  ID 

-  Today’s  date 

-  Where  in  the  test  order  the  subject  is  performing  this  test-task 

-  Inertial  sensor  location(s),  if  applicable 

-  Item  list  order  (determined  by  rolling  die  -  if  roll  number  5  first,  will  use  that  word  list 
for  the  first  memory  task  trial) 

At  task  start 

Roll  a  die  to  determine  the  order  of  the  word  lists  that  will  be  used  with  the  subject;  fill  in  the 
Trial  #  accordingly.  Trial  1  is  single  task  word  list.  Trial  2  is  the  WITHOUT  instruction  dual¬ 
task  condition.  Trials  3  and  4  are  WITH  Instruction  priority  conditions:  for  odd  ID  number 
subjects,  Trial  3  is  word  priority  and  Trial  4  is  agility  priority;  for  even  ID  number  subjects, 
Trial  3  is  agility  priority  and  Trial  4  is  word  priority. 

During  task 


Performance 

dimension 

Scoring  procedures 

Performance  subscore 

A.  Single  Task 
Condition  -  Agility 
Testing 

The  examiner  starts  the  stopwatch  when 
the  participant  is  ready  (at  start  line  in 
prone  position)  and  coincident  with  the 
“GO"  cue. 

Trial  1 :  Time  =  Time  (in  seconds)  on 
stopwatch  when  the  participant’s  first 
foot  crosses  the  end  line. 

If  participant  makes  an  error  in  the 
running  path  during  the  initial  trial,  repeat 
the  trial  in  the  single  task  conditions  (A. 
Trial  2). 


Trial  2:  Time  =  Time  (in  seconds)  on 
stopwatch  when  the  participant’s  first 
foot  crosses  the  end  line. 


B.  Single  Task 
Condition  -  Word 
List  Task/Cognitive 
Task  (Trial  #  1) 


Use  the  single  task  agility  time  from  the 
last  single  task  agility  test  as  the  “delay” 
for  single  task  word  list  testing  condition. 
After  you  present  7  words  from  the 
randomly  chosen  list,  start  the  stopwatch. 
When  the  time  for  the  agility  task  appears 
on  the  stop  watch,  ask  for  the  participant 
to  repeat  the  words  he/she  remembers. 

Record  the  words  presented  in  order  they 
are  reciting.  Write  in  any  incorrect  words 
that  are  recalled. 


If  SM  recalls  5  or  fewer  words  in  this 
task,  use  5  words  in  remaining  dual¬ 
task  trials.  If  SM  recalls  6  or  7  words 
in  this  task,  use  that  number  in 
remaining  dual-task  trials. 

Mark  this  number  on  the  first  page  of 
the  scoresheet  in  the  blank  recorded 
for  it. 

Use  this  number  as  the  word  list 
length  for  dual-task  trials.  Mark  this 
maximum  word  list  length  for  each 
relevant  trial  as  a  reminder  to  stop 
at  the  correct  number  (e.g.,  only  5 
words  presented). 


List  5 Trial#  j  single  Cl  dual 
If  dual:  P  WITH  mstr.  ^WITHOUT  ins 
D  word  priority  □  agility  priority 


1.  E-tool 


2.  Boots 


_L 


3,  Pistol 


4.  Duffel  bag 


T.lodina 


Words  Recalled  Correctly;  fT 

Word  Errors:  _£} _ 

Agility  Test  Time:  tJ  /  A  ,  ,t 
Agility  Course  Errors:  i'Vj t  - 


C.  Dual  Task 
Condition  - 
WITHOUT  priority 
instructions  (Trial 
#2) 


The  examiner  presents  the  word  list  to  the 
participant,  pauses  for  1  second,  then 
starts  trial  with  “Ready,  set,  Go” 
instruction.  Start  the  stopwatch  coincident 
with  “GO".  End  the  trial  when  the 
participant’s  first  foot  crosses  the  end  line. 
After  the  agility  task  is  completed,  the 
participant  reports  as  many  words  as 
he/she  can  remember  from  the  list.  In  the 
box  associated  with  Trial  #  2,  place  a 
number  next  to  each  word  in  the  order  in 
which  the  subject  reports  back.  Record 
any  erroneous  words  that  are  reported  as 
well.  Count  these  as  errors,  meaning  it  is 
possible  for  the  number  of  correct  words 
and  errors  to  add  to  more  than  the 
number  of  words  provided. 


In  the  box  associated  with  Trial  #2, 
in  the  following  based  on  the 
definitions  provided  earlier  in  this 
scoring  guide. 

•  Words  recalled  correctly 

•  Word  errors 

•  Agility  test  time  (to  the 
hundredth  of  a  second) 

•  Number  of  agility  course 
errors 


fill 


i 

List  3  Trial  #  u  single  ^duaf 

If  dual:  □  WITH  instr.  J£ WITHOUT  instr. 
□  word  prlori:v  □  agility  priority 

1.  Radio  Z 

2.  Flashlight  L 

3.  Gogg  les  k  *  pit? u  X 

4.  Poncho  ^  1 

A  Magazine  -4 

v 

7,  Master 

Words  Recalled  Correctly:  *4- 
Word  Errors:  ~Z- 

Agility  Test  Time: _ 

Agility  Course  Errors;  \ 

D1.  And  D2.  Dual 
Task  Conditions 
WITH  priority 
instructions  (Trials 
#  3  &  4) 


The  priority  for  each  condition  is 
described  before  the  word  list  is  shared. 
The  examiner  presents  the  word  list  to  the 
participant,  pauses  for  1  second,  then 
starts  trial  with  “Ready,  set,  Go” 
instruction.  Start  the  stopwatch  coincident 
with  “GO".  End  the  trial  when  the 
participant’s  first  foot  crosses  the  end  line. 
After  the  agility  task  is  completed,  the 
participant  reports  as  many  words  as 
he/she  can  remember  from  the  list.  In  the 
boxes  associated  with  Trials  #  3  &  4, 
place  a  number  next  to  each  word  in  the 
order  in  which  the  subject  reports  back. 
Record  any  erroneous  words  that  are 
reported  as  well. 


In  the  boxes  associated  with  Trials  #3 
&  4,  fill  in  the  following  based  on  the 
definitions  provided  earlier  in  this 
scoring  guide. 

•  Words  recalled  correctly 

•  Word  errors 

•  Agility  test  time  (to  the 
hundredth  of  a  second) 

•  Number  of  agility  course 
errors 


List  4  Tria  1  #  _2_  □  single  [S  d  ual 

If  dual:  5^  WITH  instr.  □  WITHOUT  in&tr. 
[Sword  priority  □  agility  priority 

1 .  Protractor  f 

2.  Knife  2 

3.  Bandoleer  V  X 

4.  Wat eft  3 

5.  Jacket  X 

6  Dr.UT  4 

7.  Lanyard 

Words  Recalled  Correctly:  ^ 

Word  Errors  & 

Aq  Flltv  Test  Time:  2-^.  1 1 

A  £5  i  lity  Cou  rse  E  rrors : 

IAT  Scoring  Frequently  Asked  Questions 

Word  lists 

How  should  I  score  word  list  recall  if  the  subject  reports  back  a  word  with  a  similar  meaning 
as  the  correct  word? 

Word  list  recall  must  be  exact  to  be  counted  as  correct. 

What  if  a  SM  remembers  a  word  from  the  list  after  they’ve  said  the  words  they  remember  and 
you’ve  written  their  responses  down? 

As  long  as  they  recall  the  word  correctly  before  you  go  on  to  the  next  trial,  count  the  recall  as 
correct. 

Agility  course 

How  should  I  score  Agility  Course  Errors  if  subject  forgets  to  serpentine  back  toward  the 
finish  line  but  then  self-corrects? 

Count  this  as  a  single  error.  This  often  will  slow  their  time  on  the  agility  course  significantly, 
so  the  error  will  be  accounted  for  in  time  in  addition. 


INSTRUMENTED  STAND  AND  WALK-GRID  COORDINATES  (ISAW-Grid) 


Instrumented  Stand  and  Walk  (ISAW)  -  Grid  Coordinates 
Description  and  Task  Set  Up 


Description:  The  SM  is  challenged  to  perform  the  Instrumented  Stand  and  Walk  (ISAW)  test 
(developed  by  APDM)  which  includes  instrumented  and  timed  assessment  of  quiet  standing  for 
30  seconds,  assessment  of  dynamic  stability  during  walking  for  two  7  m  (23  foot)  lengths  with  a 
180  degree  turn  at  midpoint  (Mancini  et  al  2012).  The  SM  will  next  memorize  an  8  digit 
alphanumeric  grid  coordinate  provided  within  the  context  of  a  simulated  patrol  mission  brief 
and  report  the  exact  sequence  back  to  the  examiner  after  45  seconds.  Finally,  both  the  ISAW  and 
the  grid  memorization  tasks  will  be  performed  simultaneously.  Accuracy  of  grid  coordinate 
recall,  postural  sway  area,  gait  path  variability,  and  time  to  complete  the  ISAW  (i.e.  gait  speed) 
will  be  measured  in  single  and  dual-task  conditions. 

Purpose:  This  task  will  assess  balance  and  gait  stability  as  well  as  working  memory  under  sub- 
maximal  exertion  conditions.  The  ability  to  learn  and  retain  operationally  relevant  information 
such  as  that  provided  in  this  task  while  moving  to  an  assigned  mission  location  has  relevance  to 
functional  duty  demands. 


mTBI-related  task  challenges:  Primary  •  Secondary  o 
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Source:  ISAW  methods  based  on  the  work  of  Mancini  M,  King  L,  Salarian  A,  Holmstrom  L, 
McNames  J,  and  Horak  F,  Mobility  Lab  to  Assess  Balance  and  Gait  with  Synchronized  Body-worn 
Sensors.  J  Bioengineer  &  Biomedical  Sci  2012 

Materials  and  Supplies 

•  Blue  painter's  tape  to  mark  the  initial  standing  position  of  subject’s  feet,  the  turn  point  at 
the  end  of  the  walkway  and  a  box  to  stand  in  which  is  just  past  the  start  position  for 
subject  to  stop  in  at  the  end  of  the  walk  (See  Figure  1). 

•  Clipboard  with  Score  sheet  that  has  Grid  coordinate  lists 

•  Pencil 

•  Stopwatch 

•  Opal  or  NexGen  inertial  sensor,  MobilityLab  (Opal)  software,  and  wireless  data  collection 
port  with  computer,  Opal  hand  held  controller,  www.apdm.com/mobility 

The  Opal  system,  which  is  used  to  quantify  participant  position  changes,  velocity  and 
acceleration,  consists  of  three,  wrist  watch-sized  wearable  inertial  sensors  attached  to 
participants  at  the  waist  and  on  each  lower  leg.  These  sensors  record  data  obtained  during 
testing,  which  is  down  loaded  onto  a  dedicated  laptop  computer  for  analysis  and  output. 


Test  Task  Set  Up 


•  30’  x  5’  testing  area 

•  Laptop  and  set  up  table  positioned  ~  5  feet  from  the  activity  start  point  for  ease  of 
monitoring  and  set  up. 

•  Refer  to  the  set-up  manual  from  MobilityLab  User's  Guide  for  the  specific  set  up/floor 
markings  and  distances  (Figure  2). 

Figure  1.  Subject  walking  towards  box  to  stand  in. 


Figure  2.  ISAW  distances 


30  Seconds 


U1AIA1A 


Quiet  Stance  | 


Walk  7  meters 


ISAW  -  Grid  Coordinates 
Examiner  Instructions  and  Script 

Before  testing  roll  a  die  to  randomly  pick  one  of  the  six  grid  coordinate  combinations.  Circle 
the  number  condition  on  the  scoring  sheet.  This  number  will  be  standardized  across  all 
remaining  test  conditions. 

INTRODUCTION 

This  dual-task  is  called  the  Instrumented  Stand  and  Walk  (ISAW)  -  Grid  Memorization 
Task.  It  will  assess  your  standing  posture,  walking  and  memory  within  the  context  of  a 
military  patrolling  scenario.  Accelerometers  measure  your  speed  and  position 
changes  during  the  test. 

Put  the  Opal  accelerometers  on  the  participant’s  ankles  and  around  his  or  her  waist. 

INSTRUCTIONS 

This  test  has  one  practice  trial  and  several  assessment  trials.  The  task  requires 
standing  still  and  walking,  a  memory  task,  and  then  several  trials  of  doing  both 
together. 

I  will  orient  you  to  each  condition  before  it  begins. 

Do  you  have  any  questions? 

•  Standardize  foot  placement  on  the  blue  tape  marks  which  are  marked  on  the  floor 
around  the  Opal  standardization  board  at  the  start  line. 

Practice  Stand  and  walk: 

The  walking  task  has  two  parts,  standing  followed  by  walking.  Stand  with  your  feet  on 
these  blue  tape  marks.  Your  eyes  should  be  open  and  focusing  on  the  “X”  on  the  wall, 
your  arms  at  your  sides.  When  I  say  “begin”  you’ll  stand  quietly  for  30  seconds. 

•  Point  to  the  X  marked  on  the  wall  at  the  end  of  the  room,  this  should  be  put  just  above 
eye  level  at  the  far  end  to  the  testing  room;  tape  marks  on  the  floor  should  be  set  up 
ahead  of  time  using  the  trapezoid  like  block  from  the  Opal  box). 

•  Demonstrate  the  walking  and  turn  (see  Opal  video)  by  walking  the  8-1 0  steps  before 
the  turn,  demonstrate  a  correct  pivot  turn  and  then  walk  back  toward  start  8-10  steps. 

Then  I’ll  say  “walk”.  When  you  hear  this,  walk  at  a  brisk,  comfortable  pace  to  the  tape, 
cross  the  tape,  turn  around  and  return  to  the  box  here.  (Have  a  taped  outline  of  a  box 
about  18”  square  about  3  feet  before  the  start  line — which  is  past  the  start  line  on  the  return 
trip.)  Stand  motionless  in  the  box  for  a  few  seconds  while  the  computer  finishes 
processing  the  trial.  I’ll  say  “RELAX”,  and  then  you  can  step  out  of  the  box. 

Do  you  have  any  questions? 

•  Perform  the  PRACTICE  ISAW  single  task  condition.  Use  your  stopwatch  to  time  a  30 
second  stand  and  then  give  command  “WALK”  and  have  them  stop  in  the  tape 
outlined  box  which  is  behind  the  start  line.  Total  time  is  about  45  seconds  depending 
on  their  walking  speed.  If  the  subject  does  not  perform  the  turn  correctly  (e.g.,  a  pivot 
turn  as  demonstrated),  repeat  the  turn  instructions  and  verify  their  understanding. 


1 .  Single  Task:  Motor  (Balance  and  walk  [ISAWD: 


Now  we  are  going  to  repeat  the  walking  task  a  few  times  with  the  computer  recording 
your  movement.  Do  you  have  any  questions  before  we  begin? 

•  Perform  the  ISAW  single  task  condition.  The  ISAW  program  is  started  when  you  say 
“BEGIN”  and  it  is  stopped  when  the  subject  stops  in  the  taped  square  box  after  the 
accelerometer  tracing  has  essentially  flat-lined.  After  the  “3. .  .2. . .  1  ”  countdown  on  the 
computer  screen,  say  “RELAX”  and  the  subject  returns  to  the  start  position. 

•  Perform  3  recorded  trials  in  all.  After  the  first  trial  say  “We  will  do  that  same  thing  2 
more  times”.  After  the  second  trial  say  “We  will  do  that  one  more  time”. 

•  If  the  subject  needs  a  reminder  of  the  standardized  start  position  say  “Arms  at  your 
sides,  feet  on  the  lines,  focus  your  gaze  on  the  “X”,  READY?”.... 

2.  Single  Task:  Cognitive  only  (Grid  Coordinates  Recall  and  Backbrief): 

Next  we  are  going  to  do  a  memory  task.  In  this  scenario,  you  are  a  squad  leader  in  a 
reconnaissance  unit.  Your  Commander  has  instructed  you  to  perform  a  recon 
operation  in  the  vicinity  of  grid  coordinates  that  will  be  provided.  I  will  read  2  letters 
and  6  numbers  to  you  ONLY  one  time 

During  this  task  there  are  several  rules: 

1)  Do  what  is  necessary  to  remember  the  coordinates,  but  you  may  not  repeat 
them  out  loud  or  write  them  down. 

2)  Listen  carefully,  as  you  need  to  remember  them  in  the  order  that  they  were 
given. 

After  45  seconds  I  will  say  “now”.  At  that  time  tell  me  the  letters  and  numbers  that 
you  remember. 

Do  you  have  any  questions? 

Are  you  ready?  Let’s  begin... 

•  Refer  to  score  sheet  for  the  script  and  list  of  grid  coordinates.  Use  selected 
coordinates  based  on  die  roll. 

•  Read  1  digit  per  second.  Drop  vocal  inflection  on  final  digit  to  communicate  list 
completion. 

•  Start  timing  on  the  stopwatch  approximately  Vz  second  after  you  say  the  final  grid  digit 
and  at  45  seconds  say  “NOW”. 

During  memory  recall  45  seconds  later,  the  examiner  will  write  down  the  letters  and  numbers 
vocalized  by  the  participant  on  the  scoring  sheet  in  the  order  they  are  provided.  A  maximum 
of  8/8  points  may  be  achieved  for  this  condition.  If  the  participant  does-remembers  less  than 
half  of  the  letters/numbers  correctly,  do  a  repeat  trial  to  ensure  they  understand  the 
instructions. 

3.  Dual-Task  Motor  Cognitive 

3a.  Dual  Task  (Walking  and  Grid  Recall  and  Backbrief) 


Now  we’ll  combine  the  stand  and  walk  test  with  the  GRID  recall  task.  FORGET  the  last 
GRID  coordinates  I  asked  you  to  remember.  I  would  like  you  to  REALLY  FOCUS  ON 
remembering  the  grid  coordinates  during  these  trials.  I  will  give  you  a  new  set  of  grid 
coordinates,  then  I  will  say  “Begin”  for  the  standing  portion.  After  30  seconds  I  will 
say  “walk”.  You  will  walk  the  course  as  before.  When  you  get  back  to  the  box,  state 
the  letters  and  numbers  that  you  remember.  Wait  until  I  say  “Relax”  before  stepping 
out  of  the  box.  Do  you  have  any  questions? 

•  Arms  at  your  sides 

•  Focus  your  gaze 

•  Feet  on  the  markers 

•  Remember  the  grid  coordinates  in  the  correct  order  AND  be  prepared  to  walk 
the  course  when  I  say  “Walk”  as  you  have  done  in  the  other  stand-walk 
conditions. 

•  Stop  in  the  box  and  when  I  give  the  command  “Now”,  repeat  the  coordinates 
that  you  remember.  Wait  until  I  say  “RELAX”,  before  stepping  out  of  the  box. 

•  See  the  score  sheet  for  script  and  choice  of  grid  coordinates. 

Mission  Brief:  Change  in  mission,  new  grid  coordinates  are _ 

Start  timing  on  the  stopwatch  when  you  say  “walk”.  Record  the  time  on  the  scoresheet  for 
each  trial. 

3b.  Dual  Task  (Walking  and  Grid  Brief  back) 

We  will  do  that  same  thing  2  more  times.  Please  FORGET  the  last  set  of  GRID 
coordinates  I  asked  you  to  remember. 

Refer  to  score  sheet  for  script  and  choice  of  grid  coordinates. 

3c.  Dual  Task  (Walking  and  Grid  Brief  back) 

This  is  the  last  trial.  Again,  please  FORGET  the  last  set  of  GRID  coordinates  I  asked 
you  to  remember. 

Refer  to  score  sheet  for  script  and  choice  of  grid  coordinates 
Remember...  (say  this  if  they  need  a  reminder  of  the  start  position) 

•  Arms  at  your  sides 

•  Focus  your  gaze 

•  Feet  on  the  markers 


Study  ID: 

Rater: 

Date: 

Order  #: 

Did  SM  complete  the  task?  _ Yes  _ No  (examiner  stopped) _ No  (subject  stopped) 


ISAW  Grid  Test-Task  Scoring  Form 


1.  SINGLE  TASK  MOTOR 

a.  Time  to  complete  7  meter  walk  (single  task  condition): _ (sec:XX). 

b.  Time  to  complete  7  meter  walk  (single  task  condition): _ (sec:XX). 

c.  Time  to  complete  7  meter  walk  (single  task  condition): _ (sec:XX). 


1_.  Median  time  (Middle  value  of  a.  or  b.  or  c.  here)  to  complete  7  m  walk _ (sec:XX) 

(Circle  Selected  Grid  Assignment  1-6  for  all  cognitive  conditions) 

2.  SINGLE  TASK  COGNITIVE  “The  Recon  Operation  is  in  the  vicinity  of _ " 

Assigned  Grid  Coordinate  Reported  Grid  Coordinate  (write  exactly  as  spoken) 

•  1)  Uniform  Charlie  6-1  -9-4-7-3  _ 

•  2)  Bravo  Gulf  3-9-2-4-8-7 

•  3)  Zulu  Mike  5-9- 1 -7-4-2 

•  4)  Echo  Quebec  6-5-9-3-7-2 

•  5)  Delta  Tango  4-9-7-3-9-2 

•  6)  Sierra  Oscar  4-1- 7 -9-3-8 

Number  of  grid  coordinates  accurately  recalled  in  correct  order  (single  task  cognitive  condition): 

2.  _ (Max  Score  8) 

3.  DUAL  TASK  MOTOR-COGNITIVE 

(Circle  Randomly  Selected  Grid  Assignment  1-6  for  all  cognitive  conditions) 

TRIAL  3A  “ Change  in  mission,  new  grid  coordinates  are _ ” 

Assigned  Grid  Coordinate  Reported  Grid  Coordinate  (write  exactly  as  spoken) 

•  1)  Romeo  X-Ray  3-8-2-9-5- 1  _ 

•  2)  Whiskey  Alpha  3-7 -8-2- 1  -9 

•  3)  Foxtrot  Kilo  5-8-1 -9-2-6 

•  4)  Yankee  Papa  2-7-5-8-6-2 

•  5)  November  India  3-5-4-8-5-1 

•  6)  Oscar  Hotel  7- 1  -3-9-4-2 

3a1 .  Time  to  complete  7  meter  walk  (dual  task  condition): _ (sec:XX). 

3a2.  Number  of  grid  coordinates  accurately  recalled  in  correct  order  (dual  task  condition): 
_ (Max  Score  8) 


Study  ID: 


TRIAL  3B “Change  in  mission,  new  grid  coordinates  are _ ” 

Assigned  Grid  Coordinate  Reported  Grid  Coordinate  (write  exactly  as  spoken) 

•  1)  Charlie  Tango  5-3-8-9-1 -4.  _ 

•  2)  Lima  Victor  2-4-7-5-9-1 

•  3)  Delta  Juliet  3-6- 1  -9-5-2 

•  4)  Alpha  November  2-5-3-9-4- 1 

•  5)  Yankee  Quebec  8-1-4-9-6-3 

•  6)  Papa  Bravo  4-1  -3-7 -5-2 

3b1 .  Time  to  complete  7  meter  walk  (dual  task  condition): _ (sec:XX). 

3b2.  Number  of  grid  coordinates  accurately  recalled  in  correct  order  (dual  task  condition): 
_ (Max  Score  8) 

TRIAL  3C “Change  in  mission,  new  grid  coordinates  are _ ” 

Assigned  Grid  Coordinate  Reported  Grid  Coordinate  (write  exactly  as  spoken) 

•  1 )  Mike  Sierra  4- 1-7 -9 -2-5.  _ 

•  2)  Hotel  Echo  1-5-3-0-4-6 

•  3)  Juliet  Uniform  2-5- 1  -9-3-  7 

•  4)  Kilo  Victor  8-3-5-9-2-4 

•  5)  Gulf  Whisky  9-2-5-8-3-7 

•  6)  Lima  India  2-6-9-3-5- 1 

3d .  Time  to  complete  7  meter  walk  (dual  task  condition): _ (sec:XX). 

3c2.  Number  of  grid  coordinates  accurately  recalled  in  correct  order  (dual  task  condition): 
_ (Max  Score  8) 

Grid  coordinate  scoring: 

1)  Digits  correct  if: 

•  first  or  last  digit  is  correct  if  stated  correctly  in  first  or  last  place 

•  any  digits  adjacent  to  first  or  last  digit  is  correct 

•  a  correct  sequence  of  three  or  more  anywhere  in  span 

2)  Letters  correct  -must  be  in  the  correct  position  (said  first  or  second)  and  order  to  be  counted  as 
correct. 

3)  If  subject  says  grid  coordinates  incorrectly  and  then  rapidly  corrects  him/herself,  the  corrected  version 
is  written  down  and  scored. 
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ISAW-Grid 

SCORING  INSTRUCTIONS 


Examiner  scoring  supplies/materials: 

•  Stopwatch 

•  Clipboard 

•  Pencil 

•  Subject  score  sheets  and  administration  instructions 

•  Opal  Movement  Monitor,  laptop  and  sensors  with  ankle  and  waist  straps 

•  Army  Phonetic  Alphabet  knowledge;  for  example  that  “D”  is  Delta,  “L”  is  Lima  so  when 
the  subject  says  “Delta”  you  write  down  “D”. 

At  the  start  of  the  task  the  rater  has  the  subject  set  up  with  the  sensors  in  place  (ankles  and 
waist)  and  has  the  OPAL  (ISAW)  computer  and  system  turned  on. 

Before  starting  the  task,  the  rater  fills  out  the  following: 

Subject’s  study  ID 
Your  Rater  ID 
Today’s  date 

Test  order  (testing  1st,  2nd  ...6th  of  the  test  tasks) 

Section  1)  For  the  SINGLE  TASK  MOTOR,  the  rater  times  how  long  it  takes  the  subject  to 
walk  from  the  “WALK”  verbal  signal  until  his  or  her  front  foot  crosses  the  start  line,  then  fills  in 
the  blank  la,  1b,  or  1c  on  the  score  sheet,  depending  on  trial.  The  examiner  records  the 
time  to  the  nearest  second  using  a  stopwatch  (1  .a,  1  .b,  1  .c). 

The  MEDIAN  TIME  is  the  median  or  middle  value  of  the  3  (la,  1b,  or  1c)  and  is  filled 
in  after  the  3rd  single  task  walking  trial  (BLANK  1 ). 

Section  2)  The  rater  circles  which  one  of  the  “Assigned  Grid  Coordinate”  choices  for 
SINGLE  TASK  COGNITIVE  was  presented  to  the  subject  at  the  beginning  of  the  trial 
(random  choice  by  use  of  a  dice  to  determine  grid  coordinate  1  -6  for  each  trial).  At  the  end  of 
the  elapsed  time,  the  examiner  says  “NOW”  and  the  rater  writes  exactly  what  the  subject 
says  on  the  blank  line  for  Single  Task  Cognitive  (for  example,  “BE  3  9  4  2  8  7”). 

In  Blank  2,  the  examiner  writes  the  #  of  grid  coordinates  recalled  in  the  correct  order. 

NOTE:  The  examiner  should  count  the  number  of  alpha-numeric  digits  reported  and 
scored  per  the  scoring  instructions  on  the  score  sheet  (scoring  rules  included  below  also). 

Grid  coordinate  scoring: 

1)  Digits  correct  if: 

•  first  or  last  digit  is  correct  if  stated  correctly  in  first  or  last  place 

•  any  digits  adjacent  to  first  or  last  digit  is  correct 

•  a  correct  sequence  of  three  or  more  anywhere  in  span 

2)  Letters  correct  -must  be  in  the  correct  position  (said  first  or  second)  and  order  to  be 
counted  as  correct. 

3)  If  subject  says  grid  coordinates  incorrectly  and  then  rapidly  corrects  him/herself,  the 
corrected  version  is  written  down  and  scored. 


Section  3  A,  B,  C)  The  rater  circles  which  one  of  the  “Assigned  Grid  Coordinate”  choices  for 
DUAL  TASK  MOTOR-COGNITIVE  Trial  3A  was  presented  to  the  subject  at  the  beginning  of 
the  trial.  At  the  end  of  the  elapsed  time  during  which  the  subject  is  standing  and  walking  the 
course,  the  examiner  says  “NOW”  and  writes  down  exactly  what  the  subject  says  on  the 
blank  line  for  Dual  Task  Trial  3A  (for  example  “BE  3  9  4  2  8  7”).  This  is  repeated  for  Dual 
Task  3B  and  3C. 

For  3a1 ,  3b1 , 3d  -  Record  the  amount  of  time  to  complete  the  7  meter  stand  and 
walk  to  the  nearest  second  (use  hand  held  stopwatch). 

For  3a2,  3b2,  3c2  -  Write  down  the  #  of  grid  coordinates  recalled  in  the  correct  order 
using  the  Grid  Coordinate  scoring  rules  described  above. 

After  subject  has  completed  task 

•  Name  each  data  file  on  the  OPAL  system  laptop  using  the  participant’s  unique  study 
ID  number,  task  (single  task/  dual  task)  condition,  and  trial  (1 ,2,  or  3)  (e.g., 
046#4_ST2  or  046#4_  DT3) 

•  Validate  and  save  the  Opal  recordings  at  the  completion  of  each  test  condition 

•  If  not  already  complete,  record  the  number  of  correctly  identified  grid  coordinates  in 
the  appropriate  fields  (cognitive  demand)  on  the  scoring  sheets. 

•  Ensure  all  motor  performance  scores  (times  in  seconds)  are  correctly  and  legibly 
recorded  on  the  scoring  sheets. 

•  Note  that  Dual-Task  Costs  for  motor  and  cognitive  task  performance  will  be  calculated 
automatically  within  the  data  base  spread  sheet. 

EXAMPLES  FOR  COUNTING  GRID  COORDINATE  Letters  and  Numbers: 

Subject  B  was  able  to  report  7  of  the  8  letters/digits  from  the  chosen  grid 
coordinate.  If  for  example  the  subject  is  given: 

"A  (alpha)-Z  (zulu)-4-8-l-6-2-9"  The  response  is  "A-Z-8-4-1-6-2" 

[first  two  are  correct  (A,Z),  second  two  transposed  (both  incorrect),  next  three 
correct  (1,6,2),  one  digit  omitted  (9)]. 

The  total  number  of  correct  numbers/digits  is  5  of  8. 

Subject  C  was  able  to  report  8  letters/digits  from  the  chosen  grid  coordinate.  If 
for  example  the  subject  is  given: 

"Juliet  Uniform  2-5-1-9-3-7"  The  response  is  "U-I-2-5-1-9-3-7"  (i.e., 
"Uniform-India_2-5-l-9-3-7") 

[U  is  transposed  (incorrect);  I  is  incorrect,  the  remainder  of  the  digits  are 
correct]. 

The  total  number  of  correct  numbers/digits  is  6  of  8. 


LOAD  MAGAZINE-RADIO  CHATTER 


Load  Magazine-Radio  Chatter  Listening 
Task  Description  and  Set  Up 


Description:  SM  completes  a  relatively  automatic  manual  task  choosing  from  a  bin  of  mixed 
size  dummy  rounds  (5.56  and  7.62  caliber)  and  loading  5.56  caliber  training  rounds  into 
magazines  as  fast  as  possible  both  in  a  single  and  a  dual  task  condition.  The  dual-task  condition 
requires  monitoring  radio  communication  and  verbally  announcing  when  radio  chatter  is 
relevant  to  scenario  instructions. 


Purpose:  The  purpose  of  this  task  is  to  assess  the  cost  of  a  cognitive  task  overlay  on  the  speed 
of  a  relatively  automated  upper  extremity  manual  task.  This  task  is  intended  to  challenge 
attention  allocation  (divided  attention),  sustained  attention,  executive  function,  manual 
dexterity/speed,  and  auditory  processing. 


m 


BI-related  task  challenges:  Primary  •  Secondary  o 


Cognitive 

Sensorimotor 

Physical 

Executive 

function 

Memory 

Attention 

Reactio 
n  time 

Eye  gaze 
tracking 

Scanning 

Vestibular 

Balance 

Bend  - 
lift 

Exertion 

Manual  UE 
Speed 

O 

• 

• 

Source:  Based  on  the  work  of  Cicerone  (1996)  assessing  dual  task  measures  in  persons  with 
mTBl.  Cicerone,  K.  D.  (1996).  Attention  deficits  and  dual  task  demands  after  mild  traumatic 
brain  injury.  Brain  Injury,  10[ 2),  79-89. 

Materials  and  Supplies 

•  1-gallon  open  bin  or  tub  for  holding  snap  cap  5.56  caliber  (M16)  and  7.62  caliber  (foil) 
dummy  rounds 

•  2nd  empty  bin  for  emptying  magazine(s)  to  allow  for  counting  the  number  of  rounds 
loaded 

•  100  snap  cap  dummy  rounds  (M16) 

•  50  snap  cap  dummy  rounds  (M20)  as  foils 

•  5  magazines  for  M16  caliber  weapon 

•  Computer  or  audio-player  such  as  an  I-pod  or  MP3  player 

•  Speakers  to  play  radio  chatter  audio  files  at  sufficient  volume. 

•  3  versions  of  prerecorded  ambient  mock  radio  chatter. 

•  Radio  chatter  Audio  files 

•  Cue  sheets-set  of  3  laminated  sheets  for  reminding  subjects  of  "key  words"  they  are 
responding  to  during  each  trial  (Practice,  single  task,  dual  task) 


Test  Task  Set  Up 

Space  estimate:  6x6  foot  area  with  rectangular  table  and  2  chairs 


Table  is  set  up  so  that  4  M16  magazines  are  directly  in  front  of  subject  with  the  bin  of  mixed 
rounds  either  to  right  or  left  side  depending  on  subject  preference.  Speakers  and  audio  player 
are  close  enough  to  play  the  sound  directly  in  front  of  the  subject.  (See  Figure  1.) 


Figure  1:  Load-Magazine-radio  chatter  set  up.  NOTE:  M16  dummy  rounds  are  kept  in  the  bin 
during  the  trials,  they  are  displayed  here  to  show  what  they  look  like. 


Load  Magazine-Radio  Chatter  Listening 
Examiner  Instructions  and  Script 


INTRODUCTION 

•  For  this  task  you  will  load  M-16  dummy  rounds  into  magazines  and  will  listen  to 
recorded  radio  chatter  for  specific  keywords. 

•  First  you  will  practice  both  tasks  by  themselves.  Then  you  will  perform  a 
recorded  trial  for  each  task  separately,  and  then  do  both  tasks  simultaneously. 

Allow  SM  to  practice  loading  rounds  for  10-50  seconds  or  so  as  you  are  setting  up,  if  you  are 
already  set  up,  you  can  skip  this  practice  and  just  move  straight  in  to  the  two  60  second 
trials. 

Make  sure  that  the  volume  is  turned  up  relatively  loudly  so  that  the  white  noise  is  audible  and 
slightly  distracting. 

INSTRUCTIONS 

1 .  Magazine  Loading  Practice: 

•  I  want  you  to  load  Ml  6  rounds  from  this  bin  into  these  magazines  as  FAST  as 
you  can  for  60  seconds.  I’ll  time  you  and  then  we  will  count  them.  If  one  of  the 
magazines  jams  or  gets  stiff,  just  set  it  down  and  pick  up  a  different  one.  If  you 
drop  a  round  on  the  floor  don’t  try  to  get  it,  just  pick  up  another  round  from  the 
bin  and  keep  loading.  Start  at  magazine  “1”  and  go  through  “4”.  The  magazines 
hold  30  rounds. 

•  Any  questions? 

•  Ready?... GO. 

•  Time  for  60  seconds  with  the  stopwatch  and  call  out  “STOP”. 

•  Empty  filled  magazines  into  the  padded  blue  bin  and  count  them,  recording  the  total 
rounds  loaded  on  score  sheet. 

•  Return  dummy  rounds  to  the  full  bin  and  REPEAT. 

•  Now  for  a  2nd  60  second  practice  trial,  remember  load  as  FAST  as  you  can. 

•  Ready?  ....GO 

•  Record  the  number  of  rounds  loaded  in  the  2nd  practice  trial  on  the  score  sheet. 

NOTE:  If,  over  time  the  magazines  start  malfunctioning,  move  to  using  #4  first  and  go  to  #1 . 
The  point  is  that  it  needs  to  be  with  the  same  magazines  in  single  and  dual  task  trials.  For 
example,  brand  new  magazines  are  much  stiffer  and  harder  to  fill  than  the  used  ones. 

2.  Single  Task  Magazine  Loading: 

•  Again,  I  want  you  to  load  Ml  6  rounds  as  FAST  as  you  can. 

•  This  time  I’ll  use  a  recording  that  has  radio  static  and  a  BEEP  TONE  sound. 

Start  loading  at  the  first  BEEP  and  Stop  when  it  BEEPS  again.  You  will  load  for 
a  little  more  than  2  minutes  this  time.  Remember,  if  the  magazine  jams  or  gets 
stiff,  set  it  down  and  pick  up  a  different  one.  If  you  drop  a  round,  keep  going.  We 
will  get  it  later. 

•  Do  you  have  any  questions?  Answer  all  questions. 


•  Ready?  Start  the  recording. 

After  the  2nd  BEEP  and  subject  stops  loading,  empty  filled  magazines  into  the  padded  blue 
bin  and  count  them,  recording  the  total  rounds  loaded  on  score  sheet.  Return  dummy  rounds 
to  the  full  bin. 


3. Radio  Chatter  Responses  Practice: 


Hand  the  TRAINING  Scenario  Sheet  to  the  subject  and  review  Key  Words  while  pointing 
them  out. 

•  Now  I  am  going  to  play  a  recording  of  radio  chatter  about  the  logistics  of  an  FTX 
for  several  platoons.  This  is  for  practice. 

•  You  are  a  radio  operator  monitoring  the  company  communications  network  for 
an  upcoming  “Whiskey”  Company  FTX.  Ignore  the  BEEP  tones  in  the  recording. 

•  Your  task  will  be  to  report  Specific  Key  Words  in  the  chatter  regarding  the  FTX. 


o 


o 


o 


When  WOLF  7  says  “OVER”  whenever  he  is  speaking,  Say  “CHECK” 
LOUDLY 

When  WOLF  2  says  “ROGER”  whenever  he  is  speaking,  Say 


CHECK”LO(/DLY 


“Other  people  will  say  these  words;  ONLY  respond  to  WOLF  7  and  WOLF 
2 


•  Do  you  have  any  questions?  (Answer  all  questions.) 

•  What  words  are  you  responding  to?  Make  sure  subject  is  correct.  ASK  THIS 
QUESTION  BOTH  with  the  Training  Scenario  in  view  and  again  with  the  sheet  out  of 
sight. 

•  Ready? 


After  the  recording  is  completed,  review  the  score  sheet  with  the  subject  and  tell  them  where 
their  errors  were  made. 

If  the  subject  is  1 00%  correct  or  makes  a  maximum  of  1  mistake  on  the  first  practice 
trial  and  he/she  does  not  want  to  run  through  the  practice  trial  a  2nd  time,  then  only  play  the 
TRAINING  Scenario  once. 

If  a  2nd  practice  is  needed,  replay  the  recording  and  mark  the  training  score  sheet  in 
the  second  column.  Again  review  the  score  sheet  with  the  subject  and  tell  them  where  their 
errors  were  made. 


EXAMINER  GUIDANCE/REMINDER: 

>  If  the  SM  speaks  softly  during  first  trial,  instruct  him/her  to  speak  louder 
and/or  move  closer. 

>  If  the  SM  makes  0  or  1  mistake,  he/she  does  not  need  a  2nd  practice. 

>  If  the  SM  makes  2  or  more  mistakes,  he/she  must  complete  a  2nd 
practice. 


4.  Radio  Chatter  Responses  (Test  1 :  Single  Task  Condition)  (Test_Script_1 ): 

Hand  the  TEST1  Scenario  Sheet  to  the  subject  and  review  Key  Words  while  pointing  them 
out. 

•  Now  I  am  going  to  play  a  new  recording  of  radio  chatter  about  a  different  FTX. 
Please  ignore  the  BEEPS  in  this  recording. 

•  You  are  now  monitoring  communications  for  a  “Tango”  Company  FTX.  Your 
task  is  the  same  as  in  the  practice  but  the  key  words  are  different. 


o  When  TIGER  4  says 
o  When  TIGER  7  says 


‘OVER”,  Say  “CHECK”  LOUDLY 
‘BREAK”,  Say  “CHECK”  LOUDLY 


•  Do  you  have  any  questions?  Answer  all  questions. 

•  What  words  are  you  responding  to?  Make  sure  subject  is  correct,  with  scenario 
sheet  in  view  and  again  with  it  out  of  view.  The  point  is  to  make  sure  the  last  thing  the 
subject  is  thinking  about  is  the  KEY  WORDs  and  not  about  other  off  task  topics.  This 
must  remain  consistent  throughout  all  trials. 

•  Ready? 

Start  the  tape,  and  mark  the  score  sheet. 

5.  Dual-Task:  Magazine  Loading  and  Radio  Chatter  (T est_Script_2) : 

Hand  the  TEST2  Scenario  Sheet  to  the  subject  and  review  Key  Words  while  pointing  them 

out 

•  Now  we  are  going  to  combine  loading  Ml 6  magazines  and  reporting  on  KEY 
words. 

•  Load  rounds  as  fast  as  you  can.  If  a  magazine  jams  just  set  it  down  and  start  a 
new  one.  If  you  drop  a  round,  don’t  chase  it,  just  keep  loading.  Start  loading 
with  the  first  BEEP  TONE.  Keep  loading  after  the  radio  chatter  stops  until  the  2nd 
BEEP  sounds. 

•  You  are  now  monitoring  communications  for  a  “Sierra”  Company  FTX. 

o  When  STRIKER  2  says  “OVER”,  say  "CHECK" LOUDLY 

“BREAK”,  say  “CHECK”  LOUDLY. 


o 


When  STRIKER  7  says 


Do  you  have  any  questions?  Answer  all  questions. 

What  words  are  you  responding  to?  Make  sure  subject  is  correct,  once  with 
scenario  sheet  in  front  of  them  and  once  with  it  out  of  sight. 

Ready? 


Start  the  tape  and  mark  the  score  sheet. 


Study  ID: 

Rater: 

Date: 

Order  #: 

Did  SM  complete  the  task? _ Yes  _ No  (examiner  stopped)  _ No  (subject  stopped) 

Military  Radio  Chatter  Test-Task  Scoring  Form  (Training  Script) 


Rounds  loaded  Practice  1  (60  s) 

Rounds  loaded  Practice  2  (60  s) 

Rounds  Loaded  Single  Task  (130  s) 

1. 

Rounds  Loaded  Dual  Task  (130  s) 

2. 

Correct  Responses:  “Check”  when  Wolf  7  says  “OVER”  or  Wolf  2  says  “ROGER”. 


Graded  Script 

Over/Rog 

er  Points 

Time  1 

Time  2 

Wolf  2:  Wolf  7  this  is  Wolf  2,  over. 

Wolf  1 :  Wolf  7  this  is  Wolf  1 ,  over. 

Wolf  7:  This  is  Wolf  7.  Battalion  has  authorized  Whiskey  Company  to 
commence  with  FTX  in  3  weeks.  OVER. 

Wolf  2:  This  is  Wolf  2. ROGER,  over. 

Wolf  1 :  This  is  Wolf  1 .  Copy  on  the  FTX.  Break. 

**Be  advised,  we  are  down  1  squad  and  3  vehicles  at  this  time,  over. 

Wolf  7:  This  is  Wolf  7.  Wolf  1 ,  Coordinate  with  the  NCOIC  from 
second  platoon  to  cross  level  the  troops  and  vehicles  you’ll  need  for 

the  oo  OVER. 

Wolf  1 :  This  is  Wolf  1 .  Roger  that.  Break. 

**Our  other  training  challenge  is  that  the  additional  M-16  range  we 
requested  for  the  FTX  is  already  occupied  by  C  Battery  that  week, 

over. 

Wolf  2:  This  is  Wolf  2.  ROGER. 

**Weapons  qual  is  a  challenge  for  us  too.  If  we  don’t  identify  another 
range,  two  of  our  squads  will  be  RED  on  their  training  status  for  the 
Commander’s  QTB,  over. 

Wolf  7:  This  is  Wolf  7.  Schedule  the  second  range  the  week  after  the 

FTX.  Break. 

**l  will  clear  it  with  Battalion  S-3,  OVER. 

Wolf  1 :  This  is  Wolf  1 .  Roger  that,  over. 

Wolf  2:  This  is  Wolf  2.  Copy  last  transmission.  We’ll  take  the  lead  on 
the  second  range,  over. 

Wolf  7:  This  is  Wolf  7.  Roger  that. 

**Wolf  2,  have  LT  Smith  come  see  me  for  the  range  book.  Break. 

**Also,  have  operations  contact  range  control  to  re-confirm  availability 
of  our  primary  range  during  the  FTX,  OVER. 

Wolf  1 :  This  is  Wolf  1 .  WILCO,  over. 

Wolf  2:  This  is  Wolf  2.  ROGER,  over. 

Wolf  7:  Wolf  7,  OVER  and  out. 

Totals:  5 -“over” 

3  -  “Roger” 

Possible  Distractors:  (13  of  both  Over  and  Roger  or  Break) 

Correct  /8 

Distract  /1 3 

Correct  18 

Distract  /1 3 

^continuation  of  speaker  from  prior  line 


RATER  INFORMATION: 

-Clear  area  is  correct  check  mark;  shaded  area  is  error  check  mark. 

-Indicate  in  box  if  Subject  says  “CHECK”  for  any  phrase,  before  the  next  keyword. 


Study  ID: 

Rater: 

Date: 

Order  #: 

Military.  Radio  Chatter  Grading.  Sheet  fTest  Script  1 -Single) 


Correct  Responses:  “Check”  when  Tiger  4  says  “OVER”  or  Tiger  7  says  “BREAK”. 


Graded  Script 

Over/Break  Points 

Tiger  3:  Tiger  7  this  is  Tiger  3,  over. 

Tiger  7:  Tiger  3  this  is  Tiger  7,  go  ahead,  over. 

Tiger  4:  Tiger  7  this  is  Tiger  4.  OVER. 

Tiger  7:  This  is  Tiger  7.  Next  week’s  training  exercise  will  be  conducted 
in  training  area  XZ.  BREAK. 

**Key  leaders  need  to  give  me  a  SITREP  on  their  status  by  1600  today, 

over. 

Tiger  4:  This  is  Tiger  4.  Roger,  OVER. 

Tiger  3:  This  is  Tiger  3.  Copy  last  transmission.  Break. 

**Are  we  still  covering  Class  1  for  all  phases  of  the  exercise,  over? 

Tiger  7:  This  is  Tiger  7.  Negative.  The  initial  piece  is  yours  but  resupply 
will  be  on  Tiger  4  at  Day  3,  over. 

Tiger  3:  This  is  Tiger  3.  Roger,  over. 

Tiger  4:  This  is  Tiger  4.  Copy  last  transmission.  Break. 

**Tiger  7,  can  you  verify  that  our  assets  are  also  tasked  to  provide  the 
command  group  transport  to  and  from  the  FTX,  OVER. 

Tiger  7:  This  Tiger  7.  Affirmative,  over. 

Tiger  3:  This  is  Tiger  3.  Tiger  4,  be  advised  the  DFAC  has  10  cases  of 
water  and  MRE’s  ready  for  pick  up,  over. 

Tiger  4:  This  is  Tiger  4.  Roger  that,  OVER. 

Tiger  7:  This  is  Tiger  7.  Tiger  3  also  be  prepared  to  transport  the  Class  V 
to  the  training  area  if  we  receive  final  approval  that  the  qualification 
range  is  a  “go”,  over. 

Tiger  3:  This  is  Tiger  3.  Roger  that,  over. 

Tiger  4:  This  is  Tiger  4.  Battalion  tasked  out  our  supply  vehicles  through 
the  end  of  next  week.  Break. 

**We  will  need  two  additional  vehicles  to  cover  the  resupply  mission 
while  keeping  a  vehicle  open  to  transport  the  command  group  OVER. 

Tiger  7:  This  is  Tiger  7.  Tiger  4,  coordinate  with  the  Tiger  3  to  cross  level 

2  vehicles  for  the  mission.  BREAK. 

**We  need  that  done  by  COB  today,  over. 

Tiger  4:  This  is  Tiger  4.  Roger  Top,  we  will  tap  SGT  Jones  on  that  at  the 
planning  meetina  OVER. 

Tiger  7:  This  is  Tiger  7.  Tiger  3,  what  is  your  status  on  drivers  at  this 
time?  BREAK. 

**Can  you  assist  Tiger  4  with  additional  personnel  for  his  taskers,  over? 

Tiger  3:  Tiger  7,  this  is  Tiger  3,  We’ll  cover  it  Top,  over. 

Tiger  7:  Tiger  7,  over  and  out. 

TOTALS 

3.  Correct  /  9 

4.  Distractors  /1 7 

^continuation  of  speaker  from  prior  line 


RATER  INFORMATION: 

-Clear  area  is  correct  check  mark;  shaded  area  is  error  check  mark. 

-Indicate  in  box  if  Subject  says  “CHECK”  for  any  phrase,  before  the  next  keyword. 


Study  ID: 

Rater: 

Date: 

Order  #: 

Military  Radio  Chatter  Grading  Sheet  (Test  Script  2-Dual) 
Correct  Responses:  “Check”  when  Striker  2says  “OVER”  or  Striker  7  says  “BREAK”. 


Graded  Script 

Over/Break  Points 

Striker  7:  Striker  2  this  is  Striker  7,  over. 

Striker  2:  Striker  7  this  is  Striker  2.  Go  ahead,  OVER. 

Striker  7:  This  is  Striker  7.  Day  1  Ops  at  the  FTX  will  include  establishing 
all  tactical  checkpoints  and  setting  up  the  bivouac  site.  BREAK. 

**Striker  2  NCO’s  will  teach  immediate  action  drills:  reaction  to  ambush; 
react  to  indirect  &  direct  fire,  over. 

Striker  2:  This  is  Striker  2.  Roger,  OVER. 

Striker  7:  This  is  Striker  7.  On  days  2  &  3  Striker  2  NCO’s  will  teach  their 
elements  small  unit  patrolling,  individual  &  squad  movement  techniques, 
and  first  aid  trainina.  BREAK. 

**They  will  also  cover  communications  and  UXO  training,  over. 

Striker  2:  This  is  Striker  2.  Roger,  OVER. 

Striker  1 :  This  is  Striker  1 .  We  are  responsible  for  teaching  Land  Nav  on 

day  3.  Break. 

**Can  we  move  that  block  of  instruction  today  4,  over? 

Striker  7:  This  Striker  7.  Negative,  cover  it  on  day  3but  coordinate  the  start 
time  with  Striker  2,  over. 

Striker  1 :  This  is  Striker  1  .Roger,  over. 

Striker  2:  This  is  Striker  2. Roger,  OVER. 

Striker  7:  This  is  Striker  7.  Striker  Ion  days  4  &  5  your  NCO’s  will  teach 

NBC  decon,  SALTE  reports,  and  MEDEVAC  lanes,  over. 

Striker  1 :  This  is  Striker  1  .Roger,  over. 

Striker  2:  This  is  Striker  2.  Striker  1 ,  we  have  the  training  plans  and 
materials  from  last  year’s  FTX  if  you  need  them.  Break. 

**SGT  Jones  will  be  the  POC  if  you  want  to  sign  for  the  training  materials, 

OVER. 

Striker  1 :  This  is  Striker  1  .WILCO.  Break. 

Have  SGT  Jones  set  it  to  the  side  and  we’ll  sign  for  it  later  today,  over. 

Striker  2:  This  is  Striker  2,  Roger  I’ll  let  him  know,  OVER. 

Striker  7:  This  is  Striker  7.  FTX  ends  on  dav  6.  BREAK. 

**Striker  1 ,  your  element  is  tasked  to  transport  personnel  and  training 
assets  back  to  garrison  and  will  police  and  close  the  training  site,  over. 

Striker  1 :  This  is  Striker  1  .Roger  that,  over. 

TOTALS 

5.  Correct  /9 

6.  Distractors  /1 4 

^continuation  of  speaker  from  prior  line 


RATER  INFORMATION: 

-Clear  area  is  correct  check  mark;  shaded  area  is  error  check  mark. 

-Indicate  in  box  if  Subject  says  “CHECK”  for  any  phrase,  before  the  next  keyword. 


11-5-2012,  7-24-13,  9-9-13, 10-14-13, 12-1-13 


Load  Magazine-Radio  Chatter  Dual  Task 
Scoring  Instructions 


Examiner  scoring  supplies/materials: 

•  Clipboard 

•  Pencil 

•  Subject  score  sheets 

•  Stopwatch 

•  Radio  recording  (iPod)  and  speakers 

Before  starting  the  task 
Fill  out  the  following: 

-  Subject’s  study  ID 

-  Your  Rater  ID 

-  Today’s  date 

-  Test  order  (testing  1st,  2nd  ...6th  of  the  test  tasks?) 

Count  number  of  rounds  for  each  of  2  timed  60  second  practice  trials  and  record  on  score 
sheet  at  the  top.  Note  that  this  is  to  reduce  the  practice  effect  and  provides  general 
information  to  see  if  there  is  a  major  practice  effect  in  the  number  of  rounds  loaded  (for 
example,  if  during  the  first  practice  trial  the  subject  loads  10  rounds  and  the  2nd  trial  the 
subject  loads  28  rounds,  this  information  will  aid  data  analysis). 

NOTE  that  the  ROUNDS  LOADED  information  is  all  recorded  on  the  TOP  of  the 
TRAINING  Script. 

Rounds  Loaded  ST 

Play  the  White  Noise  recording  which  has  tones  for  start/stop  of  loading  rounds.  When 
finished,  empty  the  magazines  into  the  padded  bin.  Count  the  number  of  dummy  rounds 

loaded  in  the  single  task  condition  and  record  under  1 . _ .  Return  rounds  to  the  start 

bin. 

TRAINING  SCRIPT 

Play  the  Training  Script  recording,  score  and  repeat  according  to  the  instructions.  Place  a 
check  mark  on  the  score  sheet  for  each  phrase  when  the  subject  says  “CHECK”.  Practice 
script  is  played  twice  unless  SM  is  1 00%  correct  or  up  to  1  wrong  on  1 st  trial  and  does  not 
want  a  2nd  practice  trial.  If  subject  makes  more  than  1  error  on  first  Training  Script,  then  they 
automatically  do  a  second  practice  round.  Record  the  number  of  correct  checks  (white  box) 
and  the  number  of  distractors  (shaded  box)  at  the  bottom  under  either  Trial  1  or  Trial  2  as 
appropriate.  [Note:  distractor/shaded  represent  errors.] 

TEST  SCRIPT  1 

Play  Test  Script  1  recording.  Place  a  check  on  the  score  sheet  for  each  phrase  when  the 
subject  says  “CHECK”. 

3.  _ is  the  number  of  correct  checks  (white  rows) 

4.  _ is  the  number  of  distractors  (shaded  rows) 


TEST  SCRIPT  2/  Rounds  Loaded  DT 

When  playing  Test  Script  2,  the  subject  is  also  loading  rounds.  Play  the  Test  Script  2-Dual 
recording  which  has  tones  for  start/stop  of  loading  rounds.  Place  a  check  on  the  score  sheet 
for  each  phrase  when  the  subject  says  “CHECK”.  When  finished,  empty  the  magazines  into 
the  padded  bin.  Count  the  number  of  dummy  rounds  loaded  in  the  dual  task  condition  and 

record  under  2. _ on  the  front  page  of  the  scoresheet  (at  the  top  of  the  form). 

Return  rounds  to  the  start  bin. 

5.  _ is  the  number  of  correct  checks  (white  rows) 

6.  _ is  the  number  of  distractors  (shaded  rows) 


After  subject  has  completed  task 

•  Write  down  any  comments  the  subjects  made  about  the  strategies  they  used  or  other 
comments. 


•  Calculation  of  dual-task  costs  for  motor  and  for  cognitive  costs  will  be  done  during 
analysis 

•  Return  any  dummy  rounds  to  the  start  bin 


July  26,  2013 
REV:  22Sept2014 
REV:  June  2015 


Load  Magazine-Radio  Chatter  Dual  Task 
Cue  Sheets 


Practice 


Say 


CHECK”  when  you  hear: 

Wolf  7  "OVER" 


Wolf  2 


"ROGER" 


Say  "CHECK"  when  you  hear: 
Tiger  4  "OVER" 

Tiger  7  "BREAK" 


Say 


T2 

"CHECK"  when  you  hear: 

Striker  2  "OVER" 


Striker  7 


"BREAK" 


APPENDIX  4:  MEETING  ABSTRACTS 


American  Physical  Therapy  Association  Combined  Sections  Meeting  Federal  Section  2016 
Anaheim,  CA-Symposia  Proposal- ACCEPTED  February  2016 

1)  Title:  Validation  of  the  Assessment  of  Military  Multitasking  Performance  for  Mild  TBI 

(Weightman,  Scherer,  McCulloch) 

2)  Course/session  description 

Concussed  Service  Members  (SM)  often  present  with  sensorimotor  and  cognitive  deficits  that  disrupt 
optimal  performance  of  Warrior  tasks.  Post  concussive  sequelae  can  be  subtle  but  sufficient  to  impede 
timely  return-to-duty  (RTD).  Current  best  practices  for  post  concussive  screening  rely  heavily  on 
symptom  self-report  and  single  domain  impairment  metrics  not  validated  against  the  functional  demands 
of  the  Warfighter.  Dual-task  and  multitask  methods  are  sensitive  to  subtle  cognitive  and  sensorimotor 
deficits  following  concussion,  although,  these  methods  have  not  previously  been  used  to  guide  military 
RTD  decision-making.  Validation  of  an  end-user  informed,  performance  based  assessment  battery  will 
enhance,  evidence-based  RTD  decision-making.  Led  by  Investigators  at  Courage  Kenny  Research 
Center,  a  team  of  civilian  and  military  rehabilitation  scientists  have  developed  the  Assessment  of  Military 
Multitask  Performance  (AMMP)  to  meet  this  need.  This  battery  of  six  dual -task  and  multitask  tests  can 
be  administered  and  scored  reliably.  Components  of  the  battery  distinguish  healthy  control  Active  Duty 
participants  (n=50)  from  SM  with  persistent  post-concussive  deficits  (n=47).  We  will  summarize  the 
AMMP  refinement  process,  the  challenges  of  establishing  reliable  task  metrics,  and  correlational  findings 
that  validate  AMMP  components. 

3)  Course/session  learning  objectives 

Learning  objectives  (at  least  3): 

1.  Discuss  challenges  and  successes  associated  with  dual-task  and  multitask  measurement  approaches  in  the 
Assessment  of  Military  Multitasking  Performance. 

2.  Highlighting  where  appropriate,  the  relative  contributions  of  both  instrumented  and  clinical  metrics  in 
each  of  the  AMMP  tasks  that  are  able  to  discriminate  Active  Duty  healthy  control  participants  from 
Soldiers  with  persistent  symptoms  following  concussion. 

3.  Summarize  correlational  findings  between  participant  symptoms,  performance  on  standard  neurocognitive 
metrics,  and  clinical  gaze  stability  measures  with  AMMP  performance  to  demonstrate  relative  construct 
validity  in  AMMP  tasks. 

4.  Explore  the  feasibility  of  administering  AMMP  test  clusters  to  patient  subjects  with  specific  impairment 
profiles  as  a  possible  next-step  in  RTD  screening  or  performance  testing. 

4)  REFERENCES: 

1.  Scherer  M,  Weightman  M,  Radomski  M,  Davidson  L,  McCulloch  K.  “Returning  Service  Members  to 
Duty  Following  Mild  Traumatic  Brain  Injury:  Exploring  the  Use  of  Dual-Task  and  Multitask  Assessment 
Methods”,  Phys  Ther.  2013;  93:1255-1267 
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2.  Radomski  MV,  Weightman  MM,  Davidson  LF,  Finkelstein  M,  Goldman  S,  McCulloch  KL,  Roy  TC, 
Scherer  M,  Stern  EB,.  Development  of  a  Measure  to  Inform  Return-to-Duty  Decision  Making  after  Mild 
Traumatic  Brain  Injury.  Military  Medicine,  178(3):246-253,  2013. 

3.  Catena  RD,  van  Donkelaar  P,  Chou  LS.,  Cognitive  task  effects  on  gait  stability  following  concussion.  Exp 
Brain  Res.  2007;  176:23-31. 

4.  Catena  RD,  van  Donkelaar  P,  Chou  LS.,  Altered  balance  control  following  concussion  is  better  detected 
with  an  attention  test  during  gait.  Gait  Posture.  2007;25:406-41 1. 

5.  Abernethy  B.  Dual-task  methodology  and  motor  skills  research:  some  applications  and  methodological 
constraints.  J  Human  Mov  Studies.  1988;  14101-132. 

6.  McCulloch  K.  Attention  and  dual-task  conditions:  physical  therapy  implications  for  individuals  with 
acquired  brain  injury.  J  Neurol  Phys  Ther.  2007;31:104-118. 

7.  Alderman  N,  Burgess  PW,  Knight  C,  Henman  C.  Ecological  validity  of  a  simplified  version  of  the 
Multiple  Errands  Shopping  Test.  J  Int  Neuropsychol  Soc.  2003;9:31-44. 

8.  Smith,  LB,  Radomski,  MV,  Davidson  LF,  Finkelstein,  M,  Weightman,  MM,  Scherer,  MR,  McCulloch,  K 
(2014).  Development  and  preliminary  reliability  of  a  multitasking  assessment  following  concussion. 
American  Journal  of  Occupational  Therapy,  68,  439-443. 

5)  Timed  outline  of  content  (including  breaks)  120  minutes  total 

I.  Session  overview  /Introduction  to  the  problem:  Valid  and  reliable  metrics  for  rehabilitation  providers  to 
contribute  to  RTD  decision-making  for  Service  members  with  mTBI  and  persistent  symptom  complaints 
(20  minutes) 

A.  Subtle  deficits  missed  by  standard  assessment  tools 

B.  Typical  dual-task  metrics;  application  to  militarized  test  metrics 

C.  Typical  multitask  metrics:  application  to  militarized  test  metrics. 

II.  AMMP  validation  findings 

•  Reliability  findings  and  challenges  (15  minutes) 

•  Ability  to  discriminate  healthy  versus  Service  Members  with  persistent  mTBI  symptoms  (25 
minutes) 

•  Correlation  to  standard  neurocognitive  and  dynamic  visual  acuity  tests  (20  minutes) 

•  Next  steps  in  validation  process — development  of  a  new  metric  (10  minutes) 

•  Conclusions,  lessons  learned,  overview  of  current  approaches  return-to-duty  decision-making;  an 
evolving  process  (15  minutes) 

•  Q  &  A/Discussion  (10-15  minutes) 

IVSTEP  Conference,  American  Physical  Therapy  Association 
(McCulloch,  Favorov,  Weightman),  IVSTEP-POSTER,  Columbus,  OH,  July  2016 
Non-linear  Analyses  of  Agility  Task  Performance  After  Military  Mild  Traumatic  Brain  Injury:  An 
Approach  to  Identify  Subtle  Motor  Control  Impairments 

Introduction:  Recent  advances  in  medical  science  have  highlighted  the  need  for  methods  to  quantify  the 
effects  of  mild  traumatic  brain  injury  (mTBI).  The  ability  to  determine  when  an  individual  has  recovered 
from  injury  is  a  particular  challenge.  Typical  approaches  use  neurological  examination, 
neuropsychological  tests,  standing  balance  tests  or  self-report  of  post-concussive  symptoms.  If 
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impairments  are  subtle,  these  methods  may  not  detect  impairments  that  could  have  significant 
implications  for  military  duty.  Quantitative  evaluation  of  performance  on  motor  tests  with  accelerometry 
can  be  both  practical  and  objective,  with  the  potential  to  be  highly  sensitive  to  even  subtle  neural 
abnormalities  associated  with  mTBI.  This  approach  has  been  used  with  some  success  in  a  stand  and  walk 
test  post-concussion,  but  performance  on  higher  level  mobility  tasks  (run,  obstacle  avoidance,  etc)  have  a 
stronger  relevance  for  military  training  and  combat  related  activities. 

Purpose:  We  investigated  whether  the  Illinois  Agility  Test  (IAT)  might  be  sensitive  to  subtle  mTBI 
impairments.  The  IAT  requires  a  transition  from  prone  to  standing,  running  short  distances  while 
navigating  multiple  obstacles,  changing  direction,  and  speed  accelerations/decelerations,  placing  high 
demands  on  the  sensorimotor  system. 

Subjects:  Recruited  from  active  duty  Army  service  members  aged  18-42  stationed  at  Ft  Bragg,  North 
Carolina.  Our  preliminary  sample  consisted  of  19  subjects  diagnosed  with  mTBI  who  were  undergoing 
comprehensive  outpatient  rehabilitation  at  Womack  Army  Medical  Center,  Department  of  Brain  Injury 
Medicine.  All  mTBI  subjects  were  at  least  30  days  post  injury,  with  persistent  symptoms  preventing 
return  to  duty  (RTD).  25  healthy  control  subjects  were  recruited  following  a  TBI  briefing  required  of  all 
soldiers  who  transfer  to  Ft  Bragg. 

Methods:  Two  tri-axial  accelerometers  (1  on  an  adjustable  headband,  1  on  a  belt  at  lumbar  area)  were 
worn  during  the  test,  allowing  for  continuous  wireless  transmission  of  time  series  data  by  each  of  the  6 
sensors  during  the  IAT.  Following  a  ‘walk-through’  of  the  IAT  and  a  practice  trial  to  ensure  the  task  was 
understood,  each  subject  ran  a  test  trial  at  full  speed.  The  power  spectrum  of  each  subject’s  time  series  for 
this  trial  was  computed  for  each  accelerometer  using  Fast  Fourier  transform.  Power  spectra  of  all  subjects 
were  averaged  to  identify  the  dominant  frequencies.  The  top  35  frequencies  were  used  as  the  input  to  a 
linear  Support  Vector  Machine  (SVM).  The  SVM  was  trained  to  distinguish  between  the  control  and 
mTBI  subjects.  Cross-validated  using  the  standard  leave-one-out  approach,  the  SVM  was  found  to 
correctly  classify  23  out  of  25  control  subjects  (92%  accuracy)  and  10  out  of  19  mTBI  subjects  (52.5% 
accuracy)  based  on  anterior-posterior  torso  accelerometer  readings.  The  other  5  sensors  were  less 
accurate. 

Conclusions  and  clinical  relevance:  In  a  group  of  subjects  with  chronic  mTBI,  performance  on  the  IAT 
using  non-linear  methods  provided  a  high  level  of  accuracy  for  identifying  healthy  control  subjects.  The 
mTBI  group  included  some  individuals  who  were  nearing  readiness  for  RTD  as  a  result  of  rehabilitation, 
which  may  explain  lower  accuracy  in  identifying  those  with  mTBI.  Our  group  plans  to  optimize  this 
method  on  a  larger  military  sample  that  must  perform  at  a  high  level  of  physical  function  under  conditions 
of  significant  risk.  The  ability  to  identify  subtle  movement  abnormalities  during  tactical  training  activities 
stands  to  aid  therapists  in  predicting  readiness  for  RTD  in  active  duty  military  populations. 


2015  American  Congress  of  Rehabilitation  Medicine  Dallas,  Tx 
Poster  Abstract  (October  2015) 

Title:  Inertial  sensors  detect  subtle  mobility  differences  in  soldiers  with  persistent  concussion 
symptoms:  preliminary  findings  for  the  instrumented  stand  and  walk  in  single  and  dual- task 
conditions 

Objective:  To  describe  findings  from  an  instrumented  30  second  stand,  7  meter  walk  with  180  degree 
turn  for  the  ability  to  distinguish  healthy  control  (HC),  “duty  ready”  Soldiers  from  Soldiers  undergoing 
rehabilitation  for  persistent  post-concussive  deficits  (mTBI).  Testing  was  completed  in  single  task  and 
dual-task  conditions  with  an  8  digit  alphanumeric  grid  memorization  task  as  the  cognitive  challenge. 
Design:  Convenience  sample  case-control  measurement  study. 

Setting:  Active  Duty  (AD)  US  Army  installation 

Participants:  AD  Soldiers  ages  18  -  42  y.o.  (n=  61  HC;  n=  37  mTBI). 

Interventions:  Not  applicable. 
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Main  Outcome  Measures:  The  Instrumented  Stand  and  Walk  (ISAW)  test  (Mobility  Lab,  APDM  INC), 
uses  small,  wireless  Opal™  movement  monitors  with  3D  angular  rate  sensor,  3D  accelerometer  and 
gyroscope.  Monitors  were  affixed  to  the  lumbar  area  and  ankles  to  quantify  sway,  gait,  and  rotational 
kinematics.  A  commercial  algorithm  analyzed  all  movements.  Three  trials  of  the  ISAW  were  completed 
in  single  task  and  dual-task  conditions.  Median  values  from  three -replication  sets  were  compared 
between  groups  for  sway,  turning,  and  gait  using  a  two-sided  t-test;  p  <  0.05. 

Results:  Measures  of  sway  during  quiet  stance  and  gait  at  self-selected  walking  pace  failed  to 
discriminate  groups  (p  >  0.05)..  Measures  of  turning  revealed  between  group  differences  in  both  single 
and  dual-task  conditions.  Soldiers  with  mTBI  demonstrated  longer  turn  durations  (p  <  0.001);  increased 
steps  to  complete  a  turn  (p  <  0.001),  and  decreased  peak  rotational  velocities  (p  =  0.003)  during  turns. 

The  number  of  recalled  grid  coordinates  also  distinguished  groups  (p  <  .008). 

Conclusions:  These  findings  support  the  utility  of  the  ISAW  in  both  single  and  dual-task  conditions  to 
detect  subtle  sensorimotor  deficits  in  Soldiers  with  persistent  post-concussive  deficits  during  a  180  degree 
turn. 
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Abstract  (250  words  or  less): 

Concussion/  mild  traumatic  brain  injury  (mTBI)  often  results  in  subtle  physical  and  psychological  deficits 
in  Service  members  (SM)  that  interfere  with  peak  performance  of  warrior  tasks.  Symptoms  associated 
with  mTBI  can  be  difficult  to  objectively  assess  and  commonly  complicate  retum-to-duty  (RTD)  efforts 
(Scherer  et  al.,  2013).  Dual-task  methods  have  demonstrated  sensitivity  to  subtle  cognitive  and 
sensorimotor  deficits  following  mild  brain  injury  (Catena  et  al.,  2007;  Abernathy  1988;  McCulloch  2007), 
although,  this  method  has  not  been  used  to  inform  military  RTD  decision-making.  Development  of  an 
end-user  informed  functional  assessment  is  an  important  step  in  advancing  improved,  evidence-driven 
RTD  decision  making.  Led  by  investigators  at  the  Courage  Kenny  Research  Center,  a  team  of 
interdisciplinary  civilian  and  military  rehabilitation  researchers  developed  a  performance-based 
assessment  battery  including  3  dual-task  components  to  help  inform  duty-readiness  decisions  after 
concussion/mTBI,  called  the  Assessment  of  Military  Multitask  Performance  (AMMP)  (Radomski  et  al., 
2013,  Scherer  et  al.,  2013,  Smith  et  al.,  2014).  Through  an  iterative  development  process,  the  dual-task 
components  in  the  AMMP  battery  have  demonstrated  acceptable  interrater  reliability.  Components  of  the 
battery  distinguish  Active  Duty  healthy  control  participants  (n=44)  from  SM  with  persistent  post- 
concussive  symptoms  (n=43).  We  will  summarize  the  AMMP  refinement  process  and  the  challenges  of 
establishing  reliable  task  metrics.  Validation  findings  to  date  will  also  be  shared.  Implications  of  these 
research  findings  on  test  development  for  civilian  practice  will  be  outlined. 

Learning  objectives  (at  least  3): 

Discuss  the  challenges  and  successes  with  regard  to  the  use  of  dual-task  metrics  in  the  Assessment  of 
Military  Multitasking  Performance  (AMMP). 

Summarize  findings  for  task  metrics  that  are  able  to  discriminate  healthy  control  from  Soldiers  with  a 
history  of  mTBI  receiving  services  in  a  military  TBI  rehabilitation  department. 

Summarize  correlational  findings  between  standard  neurocognitive  and  dynamic  visual  acuity  tests  and 
AMMP  dual-task  metrics  to  inform  construct  validity  of  the  AMMP. 

Describe  “lessons  learned”  for  application  to  performance-based  test  development  in  civilian  populations. 

Course  outline  (topic  title,  presenter,  time  allotment,  brief  outline  of  each  section)  90  minutes: 

•  Session  overview  /Introduction  to  the  problem:  Valid  and  reliable  metrics  for  rehabilitation 
providers  to  contribute  to  RTD  decision-making  for  soldiers  with  mTBI  residuals  (Weightman, 

15  minutes) 

o  Subtle  deficits  missed  by  standard  assessment  tools 
o  Typical  dual-task  metrics;  application  to  militarized  test  metrics 

•  AMMP  validation  findings  (Weightman  and  McCulloch,  35  minutes) 

•  Discriminate  validity 

•  Correlation  to  standard  tests 

•  Reliability  findings  and  challenges 

•  Next  steps  in  the  dual-task  validation  process — development  of  a  new  metric  (McCulloch,  10 
minutes) 
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•  Conclusions,  learnings  and  implications  for  civilian  practice  (McCulloch,  5  minutes) 

•  Q  &  A/Discussion  (All,  10  minutes) 

Diagnoses 

Concussion/mild  Traumatic  Brain  Injury 
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Abstract  (250  words  or  less): 

Concussion/  mild  traumatic  brain  injury  (mTBI)  often  results  in  subtle  physical  and  psychological  deficits 
in  Service  members  (SM)  that  interfere  with  peak  performance  of  warrior  tasks.  Symptoms  associated 
with  mTBI  can  be  difficult  to  objectively  assess  and  commonly  complicate  return-to-duty  (RTD)  efforts 
(Scherer  et  al.,  2013).  Multitask  methods  have  demonstrated  sensitivity  to  subtle  cognitive  and 
sensorimotor  deficits  following  mild  brain  injury  (Alderman  et  al.,  2003),  although,  this  method  has  not 
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been  previously  used  to  inform  military  RTD  decision-making.  Development  of  an  end-user  informed 
functional  assessment  is  an  important  step  in  advancing  improved,  evidence-driven  RTD  decision 
making.  Led  by  investigators  at  the  Courage  Kenny  Research  Center,  a  team  of  interdisciplinary  civilian 
and  military  rehabilitation  researchers  developed  a  performance-based  assessment  battery  that  includes  3 
multitask  scenarios  to  help  inform  duty -readiness  decisions  after  concussion/mTBI,  called  the  Assessment 
of  Military  Multitask  Performance  (AMMP)  (Radomski  et  al.,  2013,  Scherer  et  al.,  2013,  Smith  et  al., 
2014).  Through  an  iterative  development  process,  the  3  multitask  components  in  the  AMMP  battery  have 
demonstrated  acceptable  interrater  reliability.  Components  of  the  battery  distinguish  Active  Duty  healthy 
control  participants  (n=54)  from  SM  with  persistent  post-concussive  symptoms  (n=54).  We  will 
summarize  the  AMMP  refinement  process  and  the  challenges  of  establishing  reliable  task  metrics. 
Validation  findings  of  the  multitask  components  will  be  shared.  Implications  of  these  research  findings 
on  test  development  for  civilian  practice  will  be  outlined. 

Learning  objectives  (at  least  3): 

Discuss  the  challenges  and  successes  with  regard  to  multitask  metrics  in  the  Assessment  of  Military 
Multitasking  Performance  (AMMP). 

Summarize  findings  for  task  metrics  that  are  able  to  discriminate  healthy  control  from  Soldiers  with  a 
history  of  mTBI  receiving  services  in  a  military  TBI  rehabilitation  department. 

Summarize  correlational  findings  between  standard  neurocognitive  and  dynamic  visual  acuity  tests  and 
AMMP  multitask  metrics  to  inform  construct  validity  of  the  AMMP. 

Describe  “lessons  learned”  for  application  to  performance-based  test  development  in  civilian  populations. 

Course  outline  (topic  title,  presenter,  time  allotment,  brief  outline  of  each  section)  90  minutes: 

•  Session  overview  /Introduction  to  the  problem:  Valid  and  reliable  metrics  for  rehabilitation 
providers  to  contribute  to  RTD  decision-making  for  soldiers  with  mTBI  residuals  (Weightman, 

15  minutes) 

o  Subtle  deficits  missed  by  standard  assessment  tools 
o  Typical  multitask  metrics:  application  to  militarized  test  metrics 

•  AMMP  validation  findings  (Radomski  and  Scherer,  35  minutes) 

•  Discriminate  validity 

•  Correlation  to  standard  tests 

•  Reliability  findings  and  challenges 

•  Next  steps  in  validation  process — development  of  a  new  metric  (Radomski,  10  minutes) 

•  Conclusions,  learnings  and  implications  for  civilian  practice  (Radomski,  5  minutes) 

•  Q  &  A/Discussion  (All,  10  minutes) 

Diagnoses 

Concussion/mild  Traumatic  Brain  Injury 
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Title:  Inertial  sensors  detect  subtle  mobility  differences  in  soldiers  with  persistent  concussion 
symptoms:  preliminary  findings  for  the  instrumented  stand  and  walk 

Purpose/Hypothesis:  Imbalance  following  concussion  often  results  from  abnormal  sensory  integration 
between  visual,  somatosensory  and  vestibular  inputs.  Subtle  postural  control  and  mobility  deficits  may 
impact  performance,  safety,  and  readiness  to  return  to  duty  (RTD)  or  play  following  concussion/mild 
traumatic  brain  injury  (mTBI)  and  go  undetected  by  standard  clinical  balance  and  gait  tests.  The  purpose 
of  this  report  is  to  describe  findings  from  an  instrumented  30  second  stand  and  7  meter  walk  with  a  180 
degree  turn  for  the  ability  to  distinguish  healthy  control  (HC),  “duty  ready”  Soldiers  from  Soldiers 
undergoing  rehabilitation  for  persistent  deficits  following  mTBI. 

Subjects:  A  convenience  sample  of  34  healthy  (23  male,  age  26.9  +  5.1  years)  Soldiers  and  30  active 
duty  (all  male;  age  28.7  +  6.7  years)  Soldiers  receiving  treatment  for  persistent  post-concussive  symptoms 
at  a  military  TBI  care  center. 

Materials/Methods:  The  Instrumented  Stand  and  Walk  (ISAW)  test  (Mobility  Lab,  APDM  INC, 

Portland  OR)  uses  small,  wireless  Opal™  movement  monitors  which  contain  3D  angular  rate  sensor,  3D 
accelerometer  and  gyroscope.  Monitors  were  affixed  to  the  lumbar  area  and  lateral  ankles  to  quantify 
sway,  gait,  and  rotational  kinematics.  A  commercial  algorithm  was  used  to  analyze  all  movements  to 
include  the  initiation  and  completion  of  turns.  Three  trials  of  the  ISAW  were  completed.  Median  values 
from  three -replication  sets  were  compared  between  groups  for  the  sway  and  gait  parameters  using  a  two- 
sided  t-test;  p-values  <0.05  are  reported. 

Results:  Measures  of  sway  during  quiet  stance  and  gait  at  a  self-selected,  “comfortable”  walking  pace 
were  not  significantly  different  between  groups.  Instrumented  measures  of  turning  revealed  significant 
between  group  differences.  Soldiers  with  post-concussive  deficits  demonstrated  longer  turn  durations 
(HC:  1.58  +  0.29  seconds;  mTBI:  1.90  +  0.37  seconds,  p<.001);  increased  step  numbers  to  complete  a 
turn  (HC:  3.51  +  0.56;  mTBI:  4.07  +  0.87  steps,  p<.004),  and  decreased  peak  rotational  velocities  during 
turns  (HC:  225.73  ±  45.49%;  mTBI:  192.10  ±  35.33%,  p<.003). 

Conclusions:  These  findings  support  the  utility  of  the  ISAW  to  detect  subtle  sensorimotor  deficits  in 
Soldiers  with  persistent  post-concussive  deficits  during  a  180  degree  turn  which  involved  rapid  peak 
rotational  movements.  Analysis  of  sway  and  gait  kinematics  did  not  reveal  significant  between  groups 
differences. 

Clinical  Relevance:  Based  on  this  preliminary  analysis,  instrumented  assessment  with  inertial  sensors 
shows  promise  as  a  means  to  detect  subtle  post-concussive  differences  where  standard  clinical  measures 
of  mobility  and  balance  may  be  insufficient  in  highly  trained  military  personnel.  This  type  of 
instrumented  assessment  in  combination  with  other  functional  metrics  may  more  fully  characterize 
readiness  to  RTD  or  play. 

Keywords:  concussion,  inertial  sensors,  turning 
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PURPOSE:  The  Assessment  of  Military  Multitasking  Performance  (AMMP)  is  a  battery  of  military- 
related  functional  dual-tasks  and  multitasks  that  target  known  sensorimotor,  cognitive,  and  exertional 
vulnerabilities  after  concussion/mild  traumatic  brain  injury  (mTBI).  Once  validated,  the  AMMP  is 
intended  for  use  in  combination  with  other  metrics  to  inform  duty-readiness  decisions  in  service  members 
following  concussion.  A  dual  task  paradigm  requires  a  Soldier  to  perform  a  physical  and  a  cognitive  task 
simultaneously  in  order  to  compare  dual-task  with  single  task  performance.  A  multitask  format  requires 
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completion  of  complex  physical  and  cognitive  activities  that  approximate  real-world  military  tasks. 
Several  test  tasks  challenge  agility  or  activity  tolerance.  Initial  validation  for  the  AMMP  involves 
establishing  interrater  reliability  (IRR);  and  then  convergent/discriminant  validity  by  using  correlations  to 
neurocognitive  and  sensorimotor  tests  and  establishing  known  groups  validity  by  comparing  scores  on 
AMMP  tasks  between  healthy  control  (HC)  and  concussed  Soldiers  undergoing  rehabilitation  in  a  brain 
injury  clinic.  METHODS:  Using  a  convenience  sample  case-control  methodology  involving  test 
construction  and  evaluation,  a  data-driven  iterative  process  has  been  used  to  evaluate  the  six  AMMP  test 
tasks  for  interrater  reliability  (IRR)  by  3  person  rater  teams  comprised  of  physical  and  occupational 
therapists.  Scoring  discrepancies  identified  by  intraclass  correlation  coefficients  resulted  in  further 
clarifications  of  scoring  rules  and  scorer  training  requirements.  Ongoing  data  collection  efforts  continue 
at  Fort  Bragg  for  both  HC  and  concussed  Soldiers  with  a  goal  of  80  subjects  per  group.  RESULTS:  In 
addition  to  preliminary  HC  reliability  testing,  34  subjects  have  been  tested  to  date.  Reliability  findings 
frequently  differed  in  HC  versus  concussed  groups.  ICCs  for  task  completion  time  were  0.96-0.99  in  HC 
and  0.77  to  0.99  in  subjects  with  concussion.  Cognitive  components  for  each  of  the  3  dual-tasks,  such  as 
responding  to  key  words  in  recorded  radio  chatter  or  recalling  grid  coordinates,  demonstrated  ICCs 
between  0.64  and  0.99.  Subjects  with  concussion  typically  demonstrated  greater  number  and  range  of 
errors  than  were  seen  in  HC.  CONCLUSIONS:  Preliminary  testing  informed  modifications  in  test 
structure,  instruction,  and  scoring  to  enhance  IRR.  Development  of  measures  that  meet  military 
stakeholder  requirements  for  face  validity  and  functional  relevance  contribute  to  the  challenges  of 
development  of  a  valid  AMMP  battery.  The  consistency  of  scores  across  raters  and  the  ability  to 
discriminate  known  groups  are  fundamental  to  using  the  findings  of  the  AMMP  to  make  substantive 
recommendations  regarding  readiness  to  return  to  duty  following  concussion/mTBI. 

Funding  for  this  work  provided  by  MRMC  W81XWH-12-2-0070. 
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Title:  Preliminary  inter-rater  reliability  for  a  novel  dual-task  and  multitask  assessment  battery 
guiding  return-to-duty  in  concussed  Service  Members. 
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OBJECTIVES:  The  Assessment  of  Military  Multitasking  Performance  (AMMP)  is  a  battery  of  military- 
related  functional  dual-tasks  and  multitasks  that  target  known  sensorimotor,  cognitive,  and  exertional 
vulnerabilities  after  concussion/mild  traumatic  brain  injury  (mTBI).  Once  validated,  the  AMMP  is 
intended  for  use  in  combination  with  other  metrics  to  inform  duty -readiness  decisions  in  active  duty 
service  members  following  concussion.  The  assessment  of  inter-rater  reliability  (IRR)  provided  data 
regarding  IRR  of  individual  AMMP  tasks,  and  informed  training  requirements  needed  for  reliable  scoring 
of  the  battery.  Preliminary  IRR  findings  highlight  the  challenges  and  successes  in  development  of 
performance  based  assessments  designed  to  identify  subtle  deficits  in  highly  trained  personnel. 
METHODS:  Six  AMMP  test  tasks  were  individually  evaluated  for  IRR  by  3  person  rater  teams 
comprised  of  physical  and  occupational  therapists  who  were  the  task  developers,  with  at  least  one  rater 
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who  was  initially  unfamiliar  with  the  task.  Initial  IRR  evaluation  for  4  tasks  was  completed  on  20  healthy 
Soldiers  (HC),  and  for  2  tasks  on  12  HC.  Scoring  discrepancies  identified  by  the  statistical  analysis  using 
Krippendorf  Alpha  resulted  in  further  clarifications  of  scoring  rules  and  scorer  training  requirements. 
Tasks  were  again  tested  by  3  raters  on  11  to  13  Soldiers  undergoing  rehabilitation  following 
concussion/mTBI.  RESULTS:  Reliability  findings  frequently  differed  in  HC  versus  concussed  groups. 
For  example,  ICCs  for  task  completion  time  were  0.96-0.99  in  HC  and  0.77  to  0.99  in  subjects  with 
concussion.  Cognitive  components  for  each  of  the  3  dual-tasks,  such  as  responding  to  key  words  in 
recorded  radio  chatter  or  recalling  grid  coordinates,  demonstrated  ICCs  between  0.64  and  0.99.  Multitask 
metrics  demonstrated  variable  ICCs.  For  example,  task  completion  and  number  of  transits  during  a 
‘Charge  of  Quarters’  multitask  demonstrated  excellent  IRR  (ICCs  of  0.90  to  0.98).  IRR  calculations  were 
highly  sensitive  to  the  range  of  possible  values  with  metrics  that  involve  restricted  ranges  such  as  number 
of  errors,  cues,  or  rule  breaks,  demonstrating  variable  and  often  lower  ICCs  (ICC  range  0.13-0.85). 
Subjects  with  concussion  typically  demonstrated  greater  number  and  range  of  errors  than  were  not  seen  in 
testing  the  healthy  control  Soldiers.  CONCLUSIONS:  Preliminary  IRR  testing  informed  modifications 
in  test  instruction,  structure,  and  scoring  to  enhance  IRR.  Development  of  measures  that  meet  military 
stakeholder  requirements  for  face  validity  and  functional  relevance  contribute  to  the  complexity  of 
development  of  a  reliable  AMMP  battery.  The  consistency  of  scores  across  raters  is  fundamental  to  the 
ability  to  use  the  findings  of  the  AMMP  to  make  substantive  recommendations  regarding  readiness  to 
return  to  duty  following  concussion/mTBI. 

Funding  for  this  work  provided  by  MRMC  W81XWH-12-2-0070.  The  opinions  or  assertions  contained 
herein  are  the  private  views  of  the  authors  and  are  not  to  be  construed  as  official  or  as  reflecting  the  views 
of  the  Army  or  the  Department  of  Defense. 
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providers  are  increasingly  adopting  a  sports  medicine  model  to  manage  “tactical  athletes”  in  operational 
environments.  Because  pre-morbid  (baseline)  performance  is  not  typically  available  for  deployed  SM  as 
it  may  be  for  athletes,  clinicians  determine  duty  readiness  based  upon  the  absence  of  post-concussive 
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Warfighters.  While  similar  practices  described  within  the  sports  concussion  literature  guide  “return  to 
play”  determinations,  resolution  of  symptoms  or  improvement  of  isolated  impairments  may  be  inadequate 
to  predict  readiness  in  a  military  operational  environment.  Existing  clinical  metrics  informing  RTD 
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discrepancy  by  developing  challenging,  realistic,  and  “standards -based”  criteria  to  verify  operational 
competence.  Dual-  and  multitask  methods  have  been  described  previously  for  the  evaluation  of 
sensorimotor  and  cognitive  function  following  TBI.  These  show  promise  for  approximating  the  complex 
operational  demands  of  warfighting  and  guiding  RTD  decision  making. 

Key  Words:  Traumatic  Brain  Injury,  Outcomes  Assessment,  Sensorimotor  Performance 
3  Objectives  for  the  6  November  2013  talk: 

Upon  conclusion  of  this  45  minute  presentation,  the  learner  will  be  able  to  successfully: 

1)  Identify  common  clinical  symptoms  and  signs  (neurocognitive  or  sensorimotor)  associated  with 
concussion/  mTBI  in  an  operational  setting. 

2)  Identify  three  or  more  limitations  associated  with  current  “standard  of  care”  return  to  activity  (i.e.,  “play” 
or  “duty”)  standards. 

3)  Recognize  advantageous  characteristics  of  dual-  or  multitask  testing  techniques  for  the  determination  of 
duty  readiness  in  concussed  SM. 
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Objective/Introduction:  The  Assessment  of  Military  Multitasking  Performance  (AMMP)  is  a  battery  of 
6  military -related  tasks  that  target  known  vulnerabilities  after  concussion/mild  traumatic  brain  injury 
(mTBI).  The  goal  of  this  project  was  to  examine  inter-rater  reliability  (IRR)  of  the  AMMP  tasks. 
Methods/Strategies/Case:  AMMP  tasks  were  rated  by  3  persons  (physical  and  occupational  therapists,  at 
least  one  rater  who  was  not  involved  in  development  of  the  tasks)  in  20  uninjured  service  members  (SM) 
for  4  tasks  and  12  SM  for  2  tasks.  Scoring  discrepancies  resulted  in  further  clarifications  of  scoring  rules 
and  scorer  training  requirements.  Tasks  were  again  tested  by  3  raters  on  1 1-13  SMs  undergoing 
rehabilitation  following  concussion. 

Results:  IRR  differed  in  uninjured  versus  concussed  groups.  ICCs  for  task  completion  time  were  0.96- 
0.99  in  uninjured  SMs  and  0.77-0.99  in  SMs  with  concussion.  Cognitive  components  for  3  dual-task 
items  demonstrated  ICCs  between  0.64  and  0.99.  Multitask  ratings  had  variable  ICCs.  Time  related  ICCs 
were  high  (ICC  .90-. 99),  but  other  elements  were  sensitive  to  restricted  range  of  scores  (e.g.  number  of 
errors  made  or  cues  provided)  resulting  in  lower  ICCs  (range  0.13-0.85).  SM  post-concussion  exhibited 
more  errors,  and  errors  that  were  not  seen  in  uninjured  SMs. 

Conclusion:  IRR  testing  resulted  in  improvements  in  test  instruction,  structure,  and  scoring  to  enhance 
IRR.  Measures  that  require  rating  of  errors,  need  for  cues  during  complex  military  oriented  tasks  can  be 
rated  reliably,  but  IRR  testing  of  SM  with  concussion  is  essential.  IRR  is  acceptable  to  allow  AMMP 
findings  to  be  considered  for  return  to  duty  readiness  decisions. 
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Objective/Introduction:  The  Assessment  of  Military  Multitasking  Performance  (AMMP)  is  a  battery  of 
6  military -related  tasks  that  target  known  vulnerabilities  after  concussion/mild  traumatic  brain  injury 
(mTBI).  Beginning  in  2013,  a  team  based  at  WAMC  began  data  collection  to  validate  this  test  battery  in 
individuals  with  persistent  symptoms  after  mTBI  and  a  comparison  group  of  active  duty  service  members. 
Methods/Strategies/Case:  A  single  testing  session  of  2  1/2-3  hours  was  conducted  to  complete  intake, 
neurocognitive  testing,  dynamic  visual  acuity  testing  and  the  6  tasks  from  the  AMMP  that  involve 
militarized  dual-  and  multi-task  scenarios:  patrol,  charge  of  quarters  duty,  load  magazine/radio  chatter, 
agility/word  list  memory,  walking/grid  coordinate  memory,  run-roll-aim.  Time  and  performance  of  each 
task  was  recorded  based  on  procedures  developed  by  an  interdisciplinary  team. 

Results:  Over  the  course  of  18  months  of  testing,  we  enrolled  and  tested  50  individuals  with  mTBI  and 
51  active  duty  healthy  subjects.  Demographics  of  each  group  and  the  differences  in  ability  to  complete 
each  AMMP  task  will  be  shared  as  preliminary  indicators  of  task  ability  to  discriminate  between  the 
injured  and  non-injured  groups. 

Conclusion:  A  successful  collaboration  with  WAMC  investigators  and  University/private  hospital 
investigators  resulted  in  data  collection  on  more  than  100  participants  to  aid  in  validation  of  the  AMMP 
test  battery  to  aid  return  to  duty  decision  making  after  military  mTBI.  Additional  analyses  will  be 
completed  by  August  2015  to  inform  future  use  or  need  for  additional  study  of  the  AMMP  battery  or  its 
components. 
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assessment  methods  that  can  be  used  to  verify  functional  performance  and  opera¬ 
tional  competence  in  a  variety  of  practice  settings.  Complex  and  ecologically  valid 
assessment  techniques  incorporating  dual-task  and  multitask  methods  may  prove 
useful  in  validating  return-to-activity  requirements  in  civilian  and  military  populations. 
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Returning  Service  Members  to  Duty  Following  Mild  Traumatic  Brain  Injury 


Over  the  last  decade,  in  excess 
of  220,000  military  service 
members  have  sustained  trau¬ 
matic  brain  injury  (TBI)  (Box  1), 
resulting  in  significant  morbidity  and 
a  commensurate  degradation  of  mil¬ 
itary  operational  readiness.12  Cur¬ 
rent  criteria  to  assess  readiness  to 
return  to  duty  (RTD)  in  an  opera¬ 
tional  environment  following  mild 
TBI  (mTBI)  are  based  primarily  on 
clinical  best  practices  and  evidence 
from  the  sports  concussion  litera¬ 
ture.3-7  Although  widely  used,  it  is 
not  clear  that  existing  return-to-play 
(RTP)  guidelines  developed  for  the 
management  of  sports-related  blunt 
head  trauma  are  sufficient  to  detect 
subtle  and  potentially  duty-limiting 
effects  of  deployment-related  mTBI.8 
The  purposes  of  this  article  are  to 
provide  perspective  on  the  current 
state  of  mTBI  assessment  in  the  mil¬ 
itary  practice  environment  and  to 
introduce  alternatives  given  emerg¬ 
ing  requirements  for  more  rigorous, 
feasible,  and  ecologically  valid  meth¬ 
ods  to  guide  RTD  decision  making. 
We  propose  a  rationale  for  shifting 
the  RTD  readiness  assessment  model 
from  an  impairment-based  approach 
to  a  more  functionally  oriented  and 
standards-based  paradigm.  Finally, 
we  highlight  relevant  findings  from 
the  dual-task  and  multitask  literature 
that  support  this  proposed  approach 
to  RTD  assessment. 


Box  1.  Traumatic  Brain  Injury  (Definition) 

The  Department  of  Defense  (DoD)  defines 
traumatic  brain  injury  as  head  injury  (via 
blunt  trauma  or  barotrauma,  or  both) 
resulting  in  even  momentary  alteration  of 
consciousness,  loss  of  consciousness,  or 
posttraumatic  amnesia.  Mild  traumatic  brain 
injury  is  further  characterized  as  meeting  one 
or  more  of  the  following  criteria:  loss  of 
consciousness  for  0  to  30  minutes,  alteration 
of  consciousness  or  mental  state  for  a 
moment  or  up  to  24  hours,  and 
posttraumatic  amnesia  for  up  to  1  day. 


RTD  Following  TBI  in  the 
Deployed  Environment: 
What  Is  the  Scope  of  the 
Problem? 

According  to  Department  of  Defense 
(DoD)  estimates,  approximately 
165,000  (75%)  of  the  220,000  TBIs 
sustained  by  US  service  members 
over  the  last  decade  have  been  clas¬ 
sified  as  mild.19  Although  these  num¬ 
bers  are  significant,  recent  epidemi¬ 
ological  studies  suggest  the 
prevalence  of  head  injury  in  return¬ 
ing  service  members  may  be  even 
greater,  with  an  estimated  11.2%  to 
22.8%  of  returning  personnel  screen¬ 
ing  positive  for  mTBI  during  their 
deployment.10-14  Blast  or  explosion 
as  a  mechanism  of  injury  is  known 
to  account  for  as  much  as  78%  to 
80%  of  in-theater-related  TBI.910 
Although  evidence  suggests  recov¬ 
ery  from  blunt  head  trauma  occurs 
days  to  weeks  after  injury,  recovery 
from  blast-related  mTBI  is  less  under¬ 
stood.5  Relative  to  blunt  head 
trauma,  injuries  from  blast  exposure 
generally  result  in  a  more  compli¬ 
cated  clinical  presentation  character¬ 
ized  by  greater  frequency  of  head¬ 
ache,  facial  injury,  visual  and  hearing 
impairment,  elevated  levels  of  vestib¬ 
ular  morbidity,  and  more  severe 
posttraumatic  stress  syndrome  symp¬ 
toms.15-18  Given  the  morbidity  and 
persistent  sequelae  associated  with 
mTBI  sustained  in-theater,  there  is 
legitimate  concern  among  military 
medical  providers  and  commanders 
that  such  complexity  may  result  in  a 
more  challenging  RTD  process,  with 
direct  implications  for  operational 
readiness  of  the  fighting  force.  Fur¬ 
thermore,  with  approximately  80% 
of  military  TBIs  occurring  in  non¬ 
combat  environments,  management 
of  TBI-related  sequelae  and  their 
potential  impact  on  readiness  repre¬ 
sents  a  persistent  and  challenging 
military  health  issue  for  the  foresee¬ 
able  future.1 


RTD  Decision  Making:  A 
Page  From  the  "RTP"  Book? 
Challenges  to  RTD  Decision 
Making  in  the  Military  Practice 
Environment 

In  recent  years,  the  “tactical  athlete” 
analogy  has  increasingly  been  used 
to  describe  the  highly  functioning 
personnel  within  the  ranks  of  the 
military,  law  enforcement,  and  fire¬ 
fighting  professions.  The  description 
of  the  modern  warrior-athlete  fits 
within  a  broader  “sports  medicine 
on  the  battlefield”  concept  that 
emphasizes  early,  far-forward  man¬ 
agement  of  injured  military  service 
members  with  the  intention  to 
return  them  quickly  to  the  battle¬ 
field.  This  model  has  been  readily 
adopted  for  the  management  of  mus¬ 
culoskeletal  injury,  although  its  util¬ 
ity  for  managing  RTD  determinations 
among  service  members  with  con¬ 
cussion  has  yet  to  be  validated. 

In  the  deployed  environment,  DoD 
policy  dictates  that  physical  thera¬ 
pists  and  occupational  therapists 
administer  functional  RTD  assess¬ 
ments  of  concussed  service  mem¬ 
bers.2 3  Military  physical  therapists 
and  occupational  therapists  are  well 
suited  to  perform  these  assessments, 
given  their  existing  doctrinal  mission 
within  the  force.  Occupational  ther¬ 
apists  are  typically  key  providers  in 
concussion  care  centers  in  the 
deployed  setting  and  are  highly 
familiar  with  combat  stress  issues. 
Physical  therapists  are  assigned 
directly  to  Brigade  Combat  Teams 
and  have  the  clinical  training  to  per¬ 
form  neurologic  assessment  and 
rehabilitation.  Physical  therapists 
provide  a  broad  spectrum  of  services 
to  their  units  ranging  from  health 
promotion  and  performance  optimi¬ 
zation  to  direct-access  patient 
care.19’20 

Current  in-theater  policy  guidelines 
require  mandatory  neurological  and 
functional  evaluations  for  personnel 
exposed  to  a  specified  number  of 
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blast-related  or  blunt  trauma-related 
events.3  Additionally,  official  guid¬ 
ance  establishes  progressively  longer 
mandatory  rest  periods  for  con¬ 
cussed  service  members  following 
each  successive  incident.3  Physical 
therapists  and  occupational  thera¬ 
pists  facilitate  recovery  and  decrease 
risk  of  cumulative  injury  by  focusing 
on  early  rest  and  graded  return  to 
activity.2122 

The  sports  concussion  literature  has 
provided  a  valuable  starting  point 
from  which  to  evaluate  RTD  assess¬ 
ment  procedures  following  mTBI  in 
both  deployed  and  continental 
United  States  (CONUS)-based  clinical 
practice  environments.  However, 
after  more  than  5  years  of  military 
TBI  research,  legitimate  questions 
remain  regarding  the  sensitivity  of 
symptom-  and  impairment-based 
testing  paradigms  for  informing 
return-to-activity  decisions  in  con¬ 
cussed  service  members.23  Within 
the  military  context,  current  RTD 
decisions  are  made  by  focusing  on 
symptom  resolution,  neurocognitive 
testing,  and  clinical  balance  assess¬ 
ments  as  primary  indicators  of  duty 
readiness. 

Symptomology 

Following  a  concussive  event,  a  ser¬ 
vice  member  may  experience  a  vari¬ 
able  range  of  sensorimotor,  cogni¬ 
tive,  and  physical  sequelae  related  to 
primary  or  secondary  injuries  affect¬ 
ing  body  structure  or  function. 
These  symptoms  may  include  head¬ 
aches,  dizziness,  imbalance,  tinnitus, 
hearing  loss,  impaired  cognitive  pro¬ 
cessing,  dysexecutive  syndrome, 
musculoskeletal  pain,  or  comorbid 
stress  symptoms.24  25  Military  medi¬ 
cal  treatment  facilities,  especially 
those  in  a  deployed  setting,  are  cur¬ 
rently  challenged  to  objectively 
assess  the  spectrum  of  vulnerabilities 
associated  with  mTBI.  Department 
of  Defense  evidence-based  clinical 
practice  guidelines  neither  support 
nor  discount  reliance  on  patient  self¬ 


report  of  symptoms  for  the  manage¬ 
ment  of  mTBI.26 

Until  recently,  with  the  widespread 
adoption  of  the  Zurich  guidelines  for 
concussion  management,  symptom 
resolution  (in  the  absence  of  more 
objective  findings)  may  have  driven 
premature  RTD  decisions.21  Such 
decision  making  can  be  particularly 
challenging  in  deployed  environ¬ 
ments,  where  sensitive  and  objective 
measures  to  justify  “sidelining”  the 
service  member  often  are  unavail¬ 
able.  The  risk  of  premature  RTD  is 
further  elevated  by  the  tendency  of 
personnel  to  downplay  or  “underre¬ 
port”  symptoms  to  hasten  their 
return  to  their  unit.27  If  not  checked 
with  more  stringent  assessment  mea¬ 
sures,  the  pervasive  willingness 
within  military  culture  to  push 
through  discomfort  and  “accomplish 
the  mission”  following  concussion 
could  lead  to  an  elevated  risk  of  post 
concussive  syndrome,  increased  like¬ 
lihood  of  subsequent  exposure,  or 
greater  risk  to  self  and  members  of 
the  unit  resulting  from  the  injured 
service  member’s  diminished  situa¬ 
tional  awareness.27 

Recent  in-theater  efforts  to  increase 
the  sensitivity  of  symptom  self- 
report  under  more  challenging  and 
realistic  conditions  have  included 
the  introduction  of  a  2-minute  RTD 
exertion  test.  Similar  to  the  concept 
of  exertion  testing  in  the  sports  con¬ 
cussion  community,  service  mem¬ 
bers  with  mTBI  who  are  symptom- 
free  at  rest  or  under  light  exertion 
conditions  are  pushed  to  perform 
under  more  strenuous  (typically 
65%-  85%  of  age-predicted  maxi¬ 
mum  heart  rate)  conditions  to  probe 
for  postconcussive  symptoms.2829 
Functional  RTD  tasks  range  in  diffi¬ 
culty  from  donning  and  doffing  of 
body  armor  and  helmet  to  road 
marching  (with  a  load)  or  sprinting 
short  distances.  Variations  of  exer¬ 
tional  testing  also  have  included  the 
use  of  push-ups,  treadmill  running, 


or  step  aerobics.8  Although  thera¬ 
pists  are  directed  to  perform  func¬ 
tional  testing,  there  is  no  clear  stan¬ 
dard  for  testing  across  practice 
settings  or  branches  of  service. 

Although  not  a  “gold  standard”  diag¬ 
nostic  metric,  there  is  an  implicit 
responsibility  for  peers  and  leaders 
to  observe  and  confirm  a  service 
member’s  readiness  to  resume  duty 
when  he  or  she  returns  to  the  unit.3 
Subtle  behavioral  abnormalities  sug¬ 
gesting  persistent  mTBI-related 
impairments  often  are  first  identified 
not  by  the  service  member  or  even 
by  the  provider,  but  by  fellow  war¬ 
riors  (in  a  deployed  setting)  or  family 
members  while  at  home.13  Persistent 
postconcussive  sequelae  may  vary 
widely  and  include  difficulty  sleep¬ 
ing,  irritability,  trouble  with  peer  or 
family  relationships,  difficulty  navi¬ 
gating  uneven  or  urban  terrain  under 
dimly  lit  conditions,  or  a  diminished 
capacity  to  concurrently  accomplish 
multiple  activities  (ie,  multitask)  rel¬ 
ative  to  one’s  premorbid  capabili¬ 
ties.30  Because  unit  leadership  may 
be  among  the  first  to  identify  behav¬ 
ioral  health  systems,  unit  leadership 
can  play  an  important  role  in  initiat¬ 
ing  appropriate  management  and 
support  actions  if  such  symptoms, 
behaviors,  or  deficient  performance 
areas  are  identified. 

Clinical  Impairment  Testing 

Neurocognitive  assessment  batteries 
used  by  military  providers  and 
researchers  for  mTBI  screening, 
management,  and  monitoring 
include,  but  are  not  limited  to,  the 
Automated  Neuropsychological 
Assessment  Metrics  (ANAM)  and  the 
Immediate  Post  Concussion  Assess¬ 
ment  and  Cognitive  Testing 
(ImPACT).31’32  Neurocognitive  test¬ 
ing  has  been  recommended  for  the 
assessment  of  suspected  concussion 
in  both  civilian  and  military  practice 
settings.  However,  it  is  difficult  to 
interpret  findings,  as  there  are  no 
normative  data  for  service  members 
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in  a  deployed  setting.  Furthermore, 
these  tests  lack  face  validity  for  ser¬ 
vice  members  and  commanders  anx¬ 
ious  to  keep  “boots  on  the  ground” 
in  an  operational  setting.27’33  34 

Balance  testing  also  is  commonly 
incorporated  into  postconcussive 
evaluations,  either  independently  or 
in  conjunction  with  a  broader  multi¬ 
modal  assessment.  Although 
research  indicates  that  a  person’s 
cognitive  performance  as  measured 
by  automated  neurocognitive  testing 
typically  returns  to  normal  within  1 
week  of  a  concussive  incident,  defi¬ 
cits  in  balance  as  measured  by  the 
Balance  Error  Scoring  System  (BESS) 
or  force  platform  systems  reveal 
impairments  that  outlast  discernible 
cognitive  symptoms.435-37  Recent 
findings  confirm  significant  recovery 
time  disparities  among  the  most 
commonly  considered  RTP  indica¬ 
tors,  including  symptom  self-report, 
balance  assessment,  and  neurocogni¬ 
tive  testing,  among  concussed  ath¬ 
letes.38  Lack  of  congruency  across 
symptom,  balance,  and  neurocogni¬ 
tive  domains  casts  reasonable  doubt 
on  the  validity  of  single-domain 
assessment  measures  for  the  identifi¬ 
cation  of  duty  limiting  impairments 
in  people  with  subtle  (but  signifi¬ 
cant)  deficits.  Complex  warfighting 
tasks  represent  a  confluence  of  mul¬ 
tiple  domains  demanding  simultane¬ 
ous  functioning  from  all.  If  a  pro¬ 
vider  bases  RTD  decisions  solely 
upon  the  absence  of  isolated  impair¬ 
ments  in  a  single  domain  (without  a 
relevant  multimodal  functional 
assessment),  the  risk  of  premature 
RTD  increases.  To  date,  assessments 
of  cognition  and  balance  have  not 
been  found  to  be  predictive  of  post¬ 
concussive  symptom  development 
or  readiness  to  return  to  activity.39 
Neither  of  these  relationships  has 
been  systematically  investigated  in  a 
military  population. 


Limitations  of  Current  Clinical 
Tests  for  Military  Populations 

Existing  clinical  tests  being  used  to 
assess  injured  service  members  are 
hampered  by  psychometric  and 
practical  issues.  Clinical  measures 
used  by  deployed  physical  therapists 
and  occupational  therapists  lack  sen¬ 
sitivity  to  high-level  functional  defi¬ 
cits  revealing  ceiling  effects  when 
used  to  assess  a  highly  conditioned 
warrior  population.40  These  tests 
lack  face  validity  among  injured  ser¬ 
vice  members  and  their  leaders 
because  it  is  unclear  how  substan¬ 
dard  performance  on  an  isolated 
body  structure- based  or  function- 
based  task  (eg,  tandem  standing) 
relates  to  performance  in  one’s  role 
as  a  combatant.  The  use  of  existing 
clinical  measures  is  further  compli¬ 
cated  by  the  lack  of  normative  values 
in  the  typical  age  and  activity  range 
of  the  service  member.  Although 
there  are  many  measures  that  have 
been  demonstrated  valid  and  reliable 
to  predict  falls  or  other  adverse  out¬ 
comes  in  aging  or  clinical  popula¬ 
tions  with  more  severe  neurologic 
pathology,  such  evidence  is  lacking 
in  service  members  who  sustain 
mTBI.  Service  members  in  military 
operations  commonly  experience 
significant  physical  and  mental 
fatigue,  elevated  stress  levels,  inade¬ 
quate  or  disrupted  sleep,  and  vari¬ 
ability  in  hydration  and  nutri¬ 
tion.41-44  As  most  research  on 
natural  recovery  following  sports 
concussion  is  based  on  care  pro¬ 
vided  under  optimal  clinical  condi¬ 
tions,  it  is  unclear  how  exposure  to 
psychologically  and  physiologically 
stressful  conditions  before,  during, 
or  even  after  clearance  to  RTD  might 
affect  outcomes. 


A  Standards-Based 
Approach  to  RTD  Decision 
Making 

From  Structure  and  Function  to 
Activities  and  Participation 

The  previous  section  highlighted  a 
number  of  symptoms  and  impair¬ 
ments  believed  to  degrade  duty  read¬ 
iness.  However,  in  addition  to  symp¬ 
toms  of  physical  discomfort,  sensory 
instability,  or  disorientation,  acutely 
concussed  personnel  may  experi¬ 
ence  activity-  or  participation-level 
performance  deficits  in  previously 
highly  practiced  and  well-trained 
military  occupational  competen¬ 
cies.27’45’46  Postconcussive  activity- 
level  deficits  in  service  members,  for 
example,  may  include  impaired 
marksmanship  (stemming  from  gaze 
instability,  visual,  or  central  process¬ 
ing  deficits),  degraded  situational 
awareness  (related  to  diminished 
visual,  auditory,  or  central  cognitive 
processing  capabilities),  or  difficulty 
engaging  in  radio  communications 
(due  to  central  auditory  or  cognitive 
processing  impairments).  Such  defi¬ 
cits  likely  reflect  diffuse  involvement 
across  multiple  domains  (eg,  senso¬ 
rimotor,  cognitive,  musculoskeletal) 
and,  although  subtle  in  some  cases, 
can  clearly  have  duty-limiting  or 
even  career-limiting  implications  if 
improperly  managed.  Deficits  associ¬ 
ated  with  concussion  also  may  result 
in  participation  restrictions  (Box  2). 
Duty  limiting  barriers  to  participa¬ 
tion  may  range  from  distraction  or 
prolonged  reaction  times  during 
patrolling  by  an  infantryman,  or 
degraded  telecommunication  perfor¬ 
mance  by  a  radio  operator,  to  unsafe 
or  poorly  executed  vehicle  handling 
during  convoy  operations  by  a  truck 
driver.  Impaired  service  member  job 
performance  has  significant  implica¬ 
tions  for  safety  and  operational  effec¬ 
tiveness  for  the  individual,  unit,  and 
mission. 
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Box  2.  International  Classification  of 
Functioning ,  Disability  and  Health  (CF) 
Model  of  Functioning  and  Disability 
(Definitions) 

Body  functions  are  physiological  functions 
of  body  systems  (including  psychological 
functions). 

Body  structures  are  anatomical  parts  of 
the  body  such  as  organs,  limbs,  and  their 
components. 

Impairments  are  problems  in  body 
function  or  structure  such  as  a  significant 
deviation  or  loss. 

Activity:  qualified  as  an  individual  capacity 
(ie,  the  ability  to  execute  a  task  or  an 
action)  or  performance  (the  ability  of  the 
individual  to  perform  an  activity  in  his  or 
her  current  environment). 

Participation:  Involvement  in  a  life 
situation. 

Participation  restrictions  are  problems 
an  individual  may  experience  in 
involvement  in  life  situations. 


The  International  Classification  of 
Functioning,  Disability  and  Health 
(ICF)  model  provides  a  framework  to 
illustrate  the  complex  interplay  of 
factors,  including  the  health  condi¬ 
tion  of  concussion,  affected  body 
structure  or  body  function  systems, 
task  performance  deficits,  and  per¬ 
sonal  or  environmental  factors  that 
collectively  contribute  to  limitations 
in  duty  readiness  or  operational  com¬ 
petence  (Figure).46 

Theoretical  and  Practical  Basis 
for  a  "Standards-Based"  RTD 
Model 

The  process  of  defining  a  service 
member’s  duty  readiness  is  complex. 
Competence  as  a  warfighter 
demands  not  only  technical  prowess 
in  military  skills,  it  also  necessitates 
resilience,  self-efficacy,  the  capacity 
for  complex  thought,  and  other  per¬ 
sonal  factors  highlighted  in  the  Fig¬ 
ure,  which  are  both  abstract  and  dif¬ 
ficult  to  measure  using  conventional 
clinical  or  impairment-based  means. 
Within  the  field  of  development  eco¬ 
nomics,  Sen47  has  described  individ¬ 
ual  capabilities  as  vectors  (in  the 
mathematical  sense),  which  may  be 
summed  together  to  obtain  an 
abstract  representation  of  one’s  total 
level  of  functioning.  From  a  theoret¬ 


ical  perspective,  we  might  draw  on 
this  approach  and  conceive  of  read¬ 
iness  as  the  vector-sum  of  relevant 
military  competencies  and  other 
nonparametric  characteristics  (such 
as  the  capacity  for  complex  thinking, 
resilience,  or  even  self-efficacy) 
deemed  critical  for  mission  success. 
This  approach  acknowledges  and 
normalizes  the  heterogeneous 
nature  of  inputs  into  the  readiness 
equation  and  accounts  for  individual 
differences  in  outcomes  based  on  an 
individual’s  premorbid  capability  set 
and  coping  strategies.  Conceptually, 
this  approach  mirrors  the  complex 
contributions  to  functioning  in  the 
framework  posed  by  the  ICF  model. 

Existing  military  performance  stan¬ 
dards  require  demonstrated  compe¬ 
tence  in  warfighting  capabilities 
(ICF:  activity/participation  level), 
based  on  well-established  tasks,  con¬ 
ditions,  and  standards.48  Currently, 
clinical  decisions  guiding  RTD  fol¬ 
lowing  concussion  are  objectively 
informed  primarily  at  the  level  of 
body  structure  and  function.27  One 
might  argue  that  given  the  variability 
inherent  in  human  functioning  and 
performance,  any  attempt  to  quan¬ 
tify  a  participation  level  construct 
such  as  duty  readiness  should  be 
informed  by  activity-  or 
participation-level  performance  met¬ 
rics.  It  is  likely  that  any  advance  in 
readiness  assessment  methods  not 
recognized  as  ecologically  valid  by 
the  warfighter  community  will  fall 
short  in  key  domains  of  realism,  gen- 
eralizability,  and  complexity  neces¬ 
sary  to  determine  safe  and  appropri¬ 
ate  return  of  injured  service 
members  to  duty. 

Foundational  competencies  or  stan¬ 
dards  of  soldiering  are  described  in 
terms  of  warrior  tasks  and  battle 
drills.48  Formally  defined,  warrior 
tasks  are  a  collection  of  individual 
soldier  skills  deemed  critical  to  sol¬ 
dier  survival,  including  activity-level 
competencies  such  as  proficiency 


with  weapons  handling,  communica¬ 
tions  skills,  or  negotiating  obstacles. 
Duty  readiness  in  the  operational 
environment  also  requires  profi¬ 
ciency  with  integrated,  multiperson, 
unit-level  activities  known  as  battle 
drills .  These  participation-level  com¬ 
petencies  are  complex  “tasks  per¬ 
formed  as  a  part  of  a  unit  in  order  to 
react  and  survive  in  common  combat 
situations”  and  include  a  range  of 
activities  from  dismounted  patrolling 
to  casualty  evacuation.48  According 
to  existing  military  operational  com¬ 
petence  standards,  individual  and 
collective  service  member  profi¬ 
ciency  in  these  types  of  complex  mil¬ 
itary  tasks  are  essential  for  an  orga¬ 
nization  to  be  deemed  mission  ready. 

In  order  to  objectively  measure  ser¬ 
vice  member  performance  in  a  way 
that  is  ecologically  valid,  an  assess¬ 
ment  must  simulate  the  vocational 
demands  of  military  tasks,  demon¬ 
strate  complexity  adequate  to 
account  for  fluid  conditions  in  an 
operational  environment,  and  chal¬ 
lenge  known  mTBI-related  vulnera¬ 
bilities.  Although  the  idea  of  assess¬ 
ing  service  member  performance  on 
unmodified  warrior  tasks  to  guide 
RTD  decisions  might  be  attractive 
from  the  standpoint  of  simplicity, 
such  an  approach  can  be  problem¬ 
atic  from  a  clinical  perspective. 
Without  a  consistent  methodological 
approach,  clinicians  may  find  inter¬ 
pretation  of  performance  challeng¬ 
ing.  For  example,  if  the  tested  ser¬ 
vice  member  is  experienced,  he  or 
she  may  be  able  to  rely  on  rote  motor 
memory  even  in  the  presence  of 
residual  deficits  if  the  tested  task  is 
not  assessed  with  elements  of  com¬ 
plexity  or  unpredictability  associated 
with  a  real-world  scenario. 
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Application  of  the  ICF  Model  for  RTD  Determination 


Disorder  or  Disease 
mTBI/Concussion 

^  ^ 


Body  Structure  and  Function  Domains 
(Impairment) 

•  Musculoskeletal  System  (Weakness/Pain) 

•  Vestibular  System  (Postural/Gaze  Instability) 

•  Auditory  System  (Hearing  Loss/Tinnitus) 

•  Cognition  (Impaired  Memory  or  Executive 
Function) 


a 


Y 


Environmental  Factors 
(Comorbidity) 

•  Dangerous/Operational  Environment 
(Acute  Stress  Reaction) 

•  Sustained  Operational  Tempo 
(Sleep  Deprivation) 

•  Extreme  Heat/Humidity 
(Dehydration) 

•  Extreme  Altitudes 

(Altitude  Sickness,  Decreased  CV  fxn) 


Activities 

Participation 

("Warrior  Tasks") 

("Battle  Drills") 

(Activity  Limitation) 

(Participation  Restrictions) 

•  Maintain,  Employ,  and  Engage 

•  React  to  Contact 

With  Assigned  Weapons  System 

V  "> 

•  Establish  Security 

•  Move  Under  Fire 

•  Perform  Actions  as  a  Member  of  a 

•  Perform  Voice  Communications 

Dismounted  Patrol 

(SALTE/9-Line  MEDEVAC) 

•  Evacuate  a  Casualty 

•  Maintain  Situational  Awareness 

A 

Y 

Personal  Factors 
(Potential  Vulnerability) 

•  Previous  Blast  Exposure/Concussion  History 

(Recent  History  of  Head  Tnauma-Within  72  Hours-365  Days) 

•  Premorbid  IQ 
(Lower  ASVAB  Score?) 

•  Premorbid  Technical  and  Tactical  Proficiency 

(<Time  in  Service  vs  >  Time  in  Service/Highly  Specialized  MOS) 

•  Coping  Strategy 

(Avoidance  vs  Active  Planning  and  Accommodation ) 

•  Resilience  and  Self-Efficacy 

(Poor  Insight  Into  Deficits  and  Desire  to  Prematurely  RTD  vs 
Compliance  With  Activity  Restrictions  and  Rehabilitation  Plan  of  Care) 


Figure. 

International  Classification  of  Functioning,  Disability  and  Health  (ICF)  model  for  service  member  capabilities  and  vulnerabilities.  This 
model  highlights  service  member  capabilities  and  limitations  at  every  level  of  consideration.  Body  structure  and  function  deficits 
include  known  vulnerabilities  affecting  functioning  at  the  systems  level  and  behavior.  Activity  and  participation  blocks  summarize 
relevant  warfighting  task  skills  of  varying  complexity  among  duty-ready  service  members  who  are  healthy.  Finally,  environmental  and 
personal  factors  influencing  service  member  resilience  propose  theorized  limits  on  service  member  performance.  RTD= return  to 
duty,  mTBI  =  mild  traumatic  brain  injury,  SALTE=Size  Activity  Location  Time  Equipment  Intelligence  Report,  9-Line 
MEDEVAC= Standardized  Military  Medical  Evacuation  Request,  ASVAB=Armed  Services  Vocational  Aptitude  Battery,  MOS  =  Military 
Occupational  Specialty,  CV  fxn=cardiovascular  function. 


Complex  Task  Assessment 
Following  mTBI  in  the 
Military  Treatment 
Environment 

Although  not  yet  widely  available 
throughout  the  DoD,  preliminary 
efforts  in  select  military  treatment 
facilities  and  laboratories  to  assess 
mTBI-related  deficits  have  focused 
on  developing  realistic  duty  scenar¬ 
ios  to  challenge  service  members 
across  the  range  of  functioning  (ie, 
body  function  to  activity  level 


demands).  These  approaches 
include  highly  sophisticated,  immer¬ 
sive  virtual  reality  (VR)-  based  assess¬ 
ments;  observational,  scenario-based 
programs;  and  more  clinically  ori¬ 
ented  testing  that  draws  on  compo¬ 
nents  of  each.40’49  50 

Immersive  VR  systems  such  as  the 
Computer  Assisted  Rehabilitation 
Environment  (CAREN)  provide 
highly  sensitive,  instrumented  means 
of  assessing  physical,  sensorimotor, 


and  cognitive  performance  during 
ambulation  and  other  functional 
movements  in  a  laboratory-based 
environment.4951  Use  of  instru¬ 
mented  VR  systems  are  advanta¬ 
geous  because  they  allow  an  exam¬ 
iner  to  assess  multiple  performance 
domains  simultaneously  or  to  probe 
specific  deficits  by  manipulating  rel¬ 
evant  sensory  stimuli.  As  a  clinical 
tool,  the  CAREN  has  been  used 
extensively  within  larger  DoD  medi¬ 
cal  centers  to  assess  and  treat  duty- 
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limiting  postconcussive  deficits  in 
service  members  using  highly  realis¬ 
tic  operational  scenarios  and  com¬ 
plex  task  conditions.4951  Although 
this  type  of  RTD  approach  has  great 
versatility  and  numerous  applica¬ 
tions  for  assessing  and  managing  ser¬ 
vice  members  with  concussion, 
obvious  barriers  to  widespread  use 
include  cost;  the  requirement  for 
specialized  technical  support  to  pro¬ 
gram,  run,  and  maintain  the  system; 
and  the  relative  immaturity  of  evi¬ 
dence  to  support  generalizability  of 
“readiness”  in  a  virtual  environment 
to  “fitness  for  duty”  in  an  operational 
environment. 

In  contradistinction  to  the 
laboratory-based  VR  approach, 
recent  efforts  by  rehabilitation  pro¬ 
viders  at  military  installations  such  as 
Fort  Campbell,  Kentucky,  and  Fort 
Bragg,  North  Carolina,  have  made 
significant  progress  in  developing 
RTD  testing  modules  that  integrate 
traditional  military  training  tech¬ 
niques  with  observational  methods 
from  a  multidisciplinary  team.50 
These  scenario-based  RTD  programs 
assess  a  broad  range  of  competen¬ 
cies  ranging  from  individual  warrior 
tasks  such  as  marksmanship,  vehicle 
rollover  extrication,  and  land  naviga¬ 
tion  to  more  challenging,  small 
group- oriented  battle  drills  such  as 
VR  convoy  operations  or  simulated 
combat  lifesaver  operations.  Spe¬ 
cially  designed  assessment  modules 
challenge  service  member  perfor¬ 
mance  under  highly  realistic  and  pro¬ 
gressively  more  difficult  operational 
scenarios  designed  to  approximate 
the  real-world  stresses  of  combat.  As 
with  VR,  this  approach  has  both 
strengths  and  limitations.  Although 
anecdotal  evidence  suggests  good 
face  validity  and  favorable  RTD  gen¬ 
eralizability,  scenario-based  training 
lacks  the  precise  measurement  and 
repeatability  of  instrumented 
laboratory-based  assessment  tech¬ 
niques.  Also,  like  VR,  implementa¬ 
tion  of  this  approach  requires  signif¬ 


icant  resources,  including  costly 
technology,  substantial  logistical 
support,  a  large  dedicated  clinical 
staff,  and  numerous  staff  member 
hours  to  coordinate  and  execute. 
Thus  far,  assessment  modules  have 
not  yet  been  standardized  across 
sites,  and  test  psychometrics  have 
not  yet  been  established. 

Another  RTD  assessment  approach 
seeking  to  bridge  the  sensitivity  of 
laboratory  measures  with  the  ecolog¬ 
ical  validity  of  scenario-based  tech¬ 
niques  uses  militarized  functional 
clinical  test  tasks.  Although  many 
DoD  providers  have  sought  to  objec¬ 
tively  quantify  performance  on  spe¬ 
cific  warrior  tasks  (such  as  time  to 
don  a  protective  mask  or  time  to 
complete  a  road  march  below  a  spec¬ 
ified  symptom  severity  level),  such 
efforts  have  been  neither  standard¬ 
ized  nor  validated  and  likely  lack  the 
complexity  to  discriminate  duty 
readiness.  To  address  such  limita¬ 
tions,  recent  efforts  by  a  team  of  mil¬ 
itary  and  civilian  rehabilitation  scien¬ 
tists  have  led  to  the  development  of 
a  novel  battery  of  militarized  dual 
tasks  and  multitasks  designed  to 
challenge  known  mTBI-related  vul¬ 
nerabilities.  This  battery,  known  as 
the  Assessment  of  Military  Multitask¬ 
ing  Performance  (AMMP),  repre¬ 
sents  a  preliminary  attempt  to  incor¬ 
porate  complex  clinical  testing 
methods  into  RTD  assessment  and 
illustrates  a  potential  application  of 
the  standards-based  assessment  para¬ 
digm  in  a  clinical  environment.40 
The  AMMP  integrates  dual-task  and 
multitask  paradigms  previously 
described  in  the  literature  with  func¬ 
tional  military  requirements  to  cre¬ 
ate  individual  test  tasks  able  to  probe 
the  broad  range  of  duty-limiting 
symptoms  and  deficits  associated 
with  mTBI  (Table).52-61  Although 
the  AMMP’s  ability  to  discriminate 
duty  readiness  in  service  members 
with  mTBI  has  not  yet  been  validated 
and  the  reliability  of  the  individual 
test  tasks  has  yet  to  be  reported,  sim¬ 


ilar  procedures  have  been  success¬ 
fully  applied  in  the  assessment  of  ath¬ 
letes  with  concussion  and 
mTBI.35’36’62-64  Clinical  measures 
may  have  an  added  benefit  of  supe¬ 
rior  feasibility  in  remote  or  CONUS- 
based  military  treatment  facilities  rel¬ 
ative  to  more  resource  intensive 
approaches  described  previously. 

Given  the  importance  of  defeating 
ceiling  effects  associated  with 
impairment-based  clinical  measures, 
the  adoption  of  a  more  complex 
RTD  assessment  approach  such  as 
one  using  dual-task  and  multitask 
methods  is  appealing  for  evaluating 
service  members  with  mTBI.  Multi¬ 
task  assessment  methods  are  used 
with  success  by  clinicians  with 
patients  recovering  from  moderate 
TBI  and  mild  stroke  to  tax  multiple 
cognitive  demands.  Multitask  scenar¬ 
ios  provide  semistructured  chal¬ 
lenges  of  problem-solving  and  orga¬ 
nization  skills  required  in  daily 
routines  and  work  activities  but  have 
not  been  examined  in  mTBI.57-60 
Dual-task  activities  tested  in  labora¬ 
tory  contexts  following  mTBI  show 
impairments  when  a  combination  of 
skills  must  be  performed  simultane¬ 
ously  (eg,  cognitive  task  while  walk¬ 
ing),  even  when  symptoms  have 
apparently  “resolved.”6365  These 
same  abilities,  when  tested  sepa¬ 
rately,  appear  comparable  to  those 
of  controls  who  are  healthy,  suggest¬ 
ing  it  may  be  important  to  test  in 
dual-task  conditions  to  uncover  sub¬ 
tle  mTBI  impairments.  Dual-task  and 
multitask  approaches  provide  ways 
to  probe  activity-  and  participation- 
level  performance  in  service  mem¬ 
bers  with  mTBI,  although  military- 
specific  tasks  have  not  been 
described  in  the  literature.  In  the  fol¬ 
lowing  sections,  characteristics  and 
evidence  supporting  each  approach 
are  highlighted  to  provide  an  over¬ 
view  of  their  potential  prognostic 
utility  and  clinical  feasibility  in 
assessing  service  members  with 
mTBI. 
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Table. 

Assessment  of  Military  Multitasking  Performance  (AMMP)° 


AMMP  Task 

mTBI-Related 
Vulnerabilities/ 
Task  Demands 

Task  Description 

Assessment  Metric 

Task  Rationale 

Published  Sources 
and  Stakeholder 
Inputs  Contributing 
to  Task  Design 

Illinois  Agility 
Test  (dual 
task) 

Memory,  attention, 
dynamic  stability, 
and  agility 

Single  task  (motor):  running 
distance  of  9.1  m  (30  ft) 
with  rapid  direction 
changes  and  navigation 
of  serpentine  obstacles. 

Single  task  (cognitive): 

7-word  list  memory  task. 

Dual-task  condition:  agility 
task  and  the  memory  task 
are  done  at  the  same 
time. 

Accuracy  of  memory  recall 
and  time  to  complete 
the  agility  task  are 
measured  in  single  and 
dual-task  conditions. 
Dual-task  costs  for 
cognitive  and  motor 
components. 

Tests  of  walking  with  dual¬ 
task  performance  are 
unlikely  to  identify 
discernible  dual-task  costs. 
Service  member  demand 
for  speed  and  agility 
during  quick  maneuvers 
while  attending  to  other 
information  supports  this 
high-level  balance, 
running,  and  working 
memory  task. 

Getchell  (1979)53 
McCulloch  et  al 
(2009)55 

Hyndman  et  al 
(2006)61 

Step  initiation— 
Stroop  test 
(dual  task) 

Executive  function, 
reaction  time, 
and  balance 

Single-task  condition: 
service  member  initiates 
forward  and  backward 
steps  in  response  to  a 
vibratory  stimulus  to  the 
stepping  leg. 

Dual-task  condition: 
stepping  trials  performed 
in  conjunction  with  a 
modified  visual  Stroop 
test. 

Step  initiation  time,  foot 
lift  time,  and  step  time 
in  single-task  and  dual¬ 
task  conditions. 

Testing  paradigm  allows  for 
sensitive  measurement  of 
reaction  time,  susceptible 
to  mTBI.  Vocational 
importance  of  quick 
responsiveness  to  sensory 
stimuli  supports  this  task. 

Melzer  et  al  (2007)56 

Radio  chatter- 
magazine 
load  (dual 
task) 

Executive  function, 
attention,  and 
manual  dexterity 

Single  task  (motor):  service 
member  loads  simulated 
M-1 6  rounds  into  an 
ammunition  magazine. 

Single  task  (cognitive): 
service  member  identifies 
discrete  audio  cues  on  a 
simulated  radio 
transmission. 

Dual-task  condition:  loading 
magazine  while  listening 
to  simulated  radio 
broadcast. 

Number  of  cognitive  errors 
(omission,  commission) 
and  number  of  rounds 
loaded  in  single-task  and 
dual-task  conditions. 

A  dual-task  scenario  using  a 
manual  task  and  a 
cognitive  task 
demonstrated  mTBI 
deficits.  The  requirement 
to  hear  and  identify 
relevant  information  on  a 
tactical  network  while 
performing  bimanual 
dexterity  tasks  is 
functionally  significant. 

Cicerone  (1 996)52 

ISAW-grid 
(dual  task) 

Memory,  attention, 
gaze  stability, 
balance,  and 
dynamic  stability 

Single  task  (motor): 
instrumented  postural 
sway  and  gait  assessment. 

Single  task  (cognitive): 

8-digit  alphanumeric  grid 
coordinate  memory  task. 

Dual-task  condition: 
instrumented  sway  and 
gait  measures  while 
performing  memory  task. 

Accuracy  of  memory  recall, 
postural  sway  area,  gait 
path  variability,  and  time 
for  completion  in  single¬ 
task  and  dual-task 
conditions. 

Preliminary  testing  of 
individuals  postconcussion 
using  this  paradigm  has 
been  reported.  The 
importance  of  maintaining 
postural  and  dynamic 
stability  in  activities  of 
daily  living  is  fundamental 
to  all  other  functional 
tasks,  behaviors 
anecdotally  susceptible  to 
effects  of  blast  exposure. 
This  task  utilizes 
accelerometry,  sensitivity 
that  may  be  necessary  to 
identify  movement 
aberration  resulting  from 
mTBI. 

Mancini  et  al  (201 2)54 

(  Continued) 
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Table. 

Continued 


AMMP  Task 

mTBI-Related 
Vulnerabilities/ 
Task  Demands 

Task  Description 

Assessment  Metric 

Task  Rationale 

Published  Sources 
and  Stakeholder 
Inputs  Contributing 
to  Task  Design 

SALUTE 

(multitask) 

Executive  function, 
attention, 
memory,  visual 
scanning,  gaze 
stability,  and 
exertion 

Service  member  is 
challenged  to  gather 
information  from  video 
surveillance  recordings 
and  radio  communication 
recordings  (SALUTE)  while 
performing  a  continuous 
modified  step  test  at 
>65%  of  age-predicted 
maximum  THR. 

Accuracy /errors  of  SALUTE 
report;  ability  to 
maintain  appropriate 
exertional  load. 

The  ability  to  integrate  and 
retain  in  one's  working 
memory  visual  and 
auditory  stimuli  that  are 
operationally  significant 
under  exertion  represents 
a  high  level  of  functional 
readiness  in  a  clinical 
environment  in  a  task  that 
is  clearly  relevant  to  a 
service  member. 

Warrior  Resiliency  and 
Recovery  Center, 

Fort  Campbell, 
Kentucky 

Developed  to  address 
key  vulnerabilities 
not  addressed  with 
existing  methods 

Run,  roll,  aim 
(multitask) 

Attention,  smooth 
pursuit  tracking, 
dynamic  stability, 
exertion,  vertical 
gaze  stability, 
and  monocular 
vision 

Service  member  completes 
a  high-level  mobility  task 
with  multiple  visually 
cued  maneuvers  while 
carrying  a  simulated 
weapon. 

Rapid  start,  obstacle  (trip 
wire)  avoidance,  3-  to 
5-second  rush,  dive  to  a 
prone  position,  combat 
rolling. 

Visual  target  selection 
through  weapon  scope, 
rapid  lateral  dodging  and 
back  pedaling. 

Total  time  for  complex  task 
completion  with 
penalties  for  errors; 
accuracy  of  visual  target 
identification;  head- 
mounted  inertial  sensor 
measures  of  acceleration 
and  angular  velocity  for 
movement  components. 

The  ability  to  execute 
individual  movement 
techniques  may  provoke 
vestibular  symptoms, 
known  to  be  an  issue 
following  mTBI. 

Intermittent  visual  search 
via  weapon  scope  and  fast 
position  changes 
challenges  sensory  stability 
and  motor  performance  at 
a  high  level  of  functional 
performance  in  a  task  that 
is  clearly  relevant  to  a 
service  member. 

Warrior  Resiliency  and 
Recovery  Center, 

Fort  Campbell, 
Kentucky 

Developed  to  address 
key  vulnerabilities 
not  addressed  with 
existing  methods 

CQ  duty 
(multitask) 

Executive  function, 
memory,  and 
visual  scanning 

Service  member  organizes 
and  performs  an  array  of 
interleaving  tasks 
associated  with  a 
hypothetical  assignment 
to  staff  duty,  including 
communicating 
information  via  radio  at 
the  beginning,  middle, 
and  end  of  the  task; 
assembling  a  footstool  for 
an  injured  service 
member;  filing  a  duty  log; 
and  obtaining  additional 
information  from  wall 
charts. 

Following  directions  for 
additional  subtasks,  and 
radio  when  the  exercise  is 
completed. 

A  prospective  memory  task 
also  is  incorporated  into 
the  CQ  duty  scenario. 

Number  of  subtasks 
completed  accurately. 

Number  and  types  of 
errors  and  rule  breaks. 

Number  of  transits 
between  the  4 
workstations  to  complete 
the  task. 

Overall  performance  time 
required  to  complete  the 
task. 

This  task  requires  planning  a 
series  of  subtasks  that 
dovetail  with  each  other 
to  accomplish  the  goal  in 
the  most  efficient  way, 
requiring  executive 
function. 

Working  memory 
requirements  are 
integrated  throughout 
the  task. 

Alderman  et  al 
(2003)77 

Burgess  (2000)59 

Burgess  et  al  (2006)60 

°  mTBI  =  mild  traumatic  brain  injury;  ISAW= instrumented  stand  and  walk;  SALUTE=Size,  Activity,  Location,  Unit,  Time,  Equipment  report;  THR=target  heart 
rate;  CQ=charge  of  quarters. 


Dual-Task  Performance 

Dual-task  assessment  methods 
require  an  individual  to  perform  a 
primary  task  while  simultaneously 
performing  a  secondary  task,  with 
combined  performance  compared 


with  one’s  baseline  performance  in 
each  single-task  condition.66  In  this 
context,  a  motor  task  with  a  second¬ 
ary  cognitive  task  is  a  reasonable 
combination.  Reduction  in  perfor¬ 
mance  of  a  task  when  executed  in 


conjunction  with  a  secondary  task  is 
termed  the  dual-task  cost  (eg,  cost 
in  time  or  in  number  of  errors)  of 
performing  2  tasks  simultaneously. 
The  interpretation  of  dual-task  para¬ 
digms  follows  the  view  that  human 
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processing  resources  are  limited  and 
capacity  must  be  shared  to  accom¬ 
plish  both  tasks,  often  resulting  in 
dual-task  performance  costs.67 

Many  studies  have  revealed  accentu¬ 
ated  deficits  in  dual-task  abilities  fol¬ 
lowing  concussion  and  mTBI  during 
postural  control  tasks  acutely,  with 
impairments  sometimes  persisting 
several  months  postinjury. 353662 
These  dual-task  costs  are  signifi¬ 
cantly  greater  than  those  observed  in 
age-matched  controls  and  are  influ¬ 
enced  by  environmental  and  visu- 
ospatial  complexity.62’6568-70 

The  ability  to  do  2  tasks  at  once  is 
theorized  to  require  executive  con¬ 
trol.  Attention  must  be  allocated 
appropriately  to  perform  both  tasks 
successfully.  Laboratory  studies 
using  cognitive  dual  tasks  reveal 
slower  reaction  and  response  times 
and  increased  cognitive  task  error 
following  sports  concussions.70-72 
Additionally,  difficulty  with  dual 
tasks  or  an  inability  to  perform  such 
tasks  is  associated  with  safety  prob¬ 
lems  and  may  not  be  evident  if  motor 
or  cognitive  tasks  are  assessed  singly 
and  not  in  combination.62  65  Individ¬ 
uals  with  concussion  and  mTBI  and 
those  with  more  severe  acquired 
brain  injury  show  consistent  diffi¬ 
culty  with  dual-task  performance  of 
cognitive  and  motor  tasks  in  labora¬ 
tory  dual-task  paradigms  and  clinical 
tests  during  walking.677073  After 
concussion,  dual-task  costs  have 
been  documented  in  walking  speed, 
variability,  and  stability.  The  ability 
to  orient,  allocate  attention  to,  and 
switch  focus  between  visual  stimuli 
is  impaired,  which  is  correlated  with 
problems  with  obstacle  avoidance 
while  walking.62-64’707475  Higher- 
level  balance  deficits,  vestibular 
injury,  or  musculoskeletal  injury  may 
contribute  to  these  performance 
problems.  These  dual-task  gait  defi¬ 
cits  have  been  observed  to  persist 
over  longer  time  frames  than  cogni¬ 
tive  deficits  after  concussion  and 


could  influence  mobility  on  uneven 
terrain.35  76 

Dual  tasks  that  have  been  used  clin¬ 
ically  include  memory  tasks  exe¬ 
cuted  during  walking  and  running 
conditions.  One  example  of  a  dual 
task  formulated  to  challenge  a  mili¬ 
tary  service  member  population 
could  involve  administering  the  Illi¬ 
nois  Agility  Test  (which  requires 
rapid  direction  changes  and  obstacle 
avoidance,  consistent  with  service 
member  physical  training  activities) 
while  performing  a  secondary  cogni¬ 
tive  task  to  challenge  dynamic  stabil¬ 
ity,  agility,  and  cognitive  function 
simultaneously.40  Most  studies  of 
dual-task  performance  postconcus¬ 
sion  also  have  used  sensitive  instru¬ 
mentation  to  capture  what  are  some¬ 
times  small  changes  in  postural 
control.  Dual-task  scenarios  tailored 
to  service  members  could  be 
designed  in  a  similar  way  by  using 
compact  technologies  (eg,  inertial 
sensory  measures)  to  improve  mea¬ 
sure  sensitivity  in  forward-deployed 
or  remote  environments  where  safe 
and  timely  RTD  decisions  are  most 
critical. 

Multitask  Observational 
Performance 

Competence  in  everyday  life 
requires  the  ability  to  multitask, 
using  multiple  cognitive  and  motor 
abilities  to  plan,  organize,  and  carry 
out  complex  tasks  (Box  3).  Standard¬ 
ized  testing  of  multitask  perfor¬ 
mance  is  used  in  occupational 
therapy  and  neuropsychology  to 
approximate  the  demands  of  a  real- 
world  environment  (ie,  role  engage¬ 
ment)  and  is  valued  for  its  ecological 
validity.5760  Planning,  organizing, 
and  problem  solving,  governed  by 
executive  function,  are  required  dur¬ 
ing  a  multitask  assessment.  The  eval¬ 
uator  observes  performance  for 
errors  in  action  while  a  patient  is 
given  free  rein  to  perform  prescribed 
multistep  everyday  tasks  that  involve 


an  array  of  multiple  objects,  task 
demands,  and  rules.57 


Box  3.  Burgess'  Definition  of  Multitasking 
describes  5  features  that  are 
commonly  included  in  performance- 
based  multitask  assessments. 

Many  tasks:  Numerous  separate  and  varied 
tasks  are  completed. 

Interleaving:  Tasks  are  dovetailed  (ie, 
alternated  or  coordinated  in  accordance 
with  a  plan). 

Only  one  task  performed  at  a  time: 

Tasks  are  performed  one  at  a  time  due  to 
either  cognitive  or  physical  constraints, 
further  reinforcing  interleaving. 

Interruptions  and  unexpected 
outcomes:  Tasks  are  dynamic  and  may 
have  unanticipated  interruptions  or 
situations  where  things  do  not  go  as 
originally  planned. 

Delayed  intentions:  Tasks  require  a  person 
to  remember  to  do  a  second  thing, 
unrelated  to  the  successful  completion  of 
the  overall  multitasks  (referred  to  as  a 
"prospective  memory"  requirement). 


Performance-based  multitask  assess¬ 
ments  have  been  developed  that 
focus  on  frontal  lobe  dysfunction 
that  occurs  with  stroke  and  TBI.5777 
These  assessments  reveal  common 
problems  with  multitasking  across 
the  spectrum  of  patients  with  neuro¬ 
logic  involvement  from  subtle  defi¬ 
cits  after  mild  stroke  to  more  signif¬ 
icant  cognitive  deficits  following 
moderate  to  severe  TBI.65’67’7078-80 
Without  exception,  the  multitask 
scenarios  described  in  the  literature 
lack  face  validity  for  the  military  pop¬ 
ulation;  they  require  instrumental 
activities  of  daily  living  such  as  sim¬ 
ple  cooking  tasks  or  telephone  use 
(Naturalistic  Action  Test  [NAT], 
Executive  Function  Performance 
Test),  wrapping  a  present  (NAT),  or 
running  errands  in  a  mall  or  hospital 
setting  (Multiple  Errands  Test). 
Although  these  assessments  evaluate 
high-level  executive  functioning  def¬ 
icits  and  require  prioritization  of 
tasks,  switching  sets,  and  prospec¬ 
tive  memory,  such  metrics  are  not 
reflective  of  military  vocational 
demands. 

Effective  multitasking  is  essential 
during  combat  operations.  A  report 
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by  Fischer  and  Mautone81  on  multi¬ 
tasking  requirements  in  military  envi¬ 
ronments  suggests  that  environ¬ 
ments  vary  along  3  main  dimensions: 
type  of  multitask  required  (decision 
making,  information  monitoring,  and 
task-flow  management),  intensity  of 
multitask,  and  consequences  of  fail¬ 
ure.  Multiple  sensory,  motor,  and 
cognitive  systems  contribute  to  suc¬ 
cessful  multitasking  skills,  systems 
that  may  be  compromised  following 
mTBI. 

Service  members  may  perform  well 
on  impairment-based  assessments 
that  evaluate  single-component  pro¬ 
cesses  in  nondistracting  and  non¬ 
stressful  environments.  Performance 
deficits  become  evident  when  tasks 
are  presented  with  less  structure  and 
increasing  difficulty,  requiring  real¬ 
time  decision  making  and  the  effec¬ 
tive  allocation  of  cognitive,  physical, 
and  sensorimotor  resources  across 
multiple  simultaneous  demands. 
Anecdotally,  service  members  who 
are  successful  in  performing  isolated 
cognitive,  physical,  and  sensorimo¬ 
tor  tasks  (eg,  BESS,  ANAM,  ImPACT) 
often  report  a  sense  of  feeling  “off” 
when  similar  challenges  combine 
within  the  multidimensional 
demands  that  are  critical  to  most  ser¬ 
vice  members’  duties  or  to  complex 
family  life  situations  when  in 
garrison. 

Theorized  military  multitask  scenar¬ 
ios  should  focus  on  the  multisystem 
vulnerabilities  associated  with  con¬ 
cussion  and  mTBI.  Examples  of  mul¬ 
titask  formulations  that  may  prove 
useful  in  discriminating  RTD  readi¬ 
ness  have  recently  been  described.40 
One  such  measure  challenges  a  ser¬ 
vice  member  to  observe,  process, 
and  retain  relevant  information  from 
a  customized,  computer-generated 
mission  scenario  while  continuously 
stepping  on  an  exercise  step  at  a 
moderate  pace.  This  task  combines 
physical  exertion  with  a  demand  for 
vigilance  or  “situational  awareness” 


during  a  simulated  dismounted 
patrol  in  a  way  that  approaches  the 
real-world  demands  on  a  member  of 
a  reconnaissance  patrol  in  deployed 
environment.  Although  highly  realis¬ 
tic  computer  graphics  and  meticu¬ 
lously  scripted  scenario  content 
allow  an  examiner  to  target  known 
mTBI-related  vulnerabilities,  this 
assessment  differs  from  more  sophis¬ 
ticated  VR  approaches  in  its  simplic¬ 
ity  and  clinical  feasibility.  The  task 
can  be  projected  to  any  treatment 
environment  that  will  support  a 
computer  monitor  and  an  exercise 
step  (with  or  without  inertial  sensor 
data  collection).  Another  task 
approximates  the  physical  agility 
required  for  military  individual 
movement  techniques  while  inter¬ 
mittently  challenging  visual  sensory 
stability  and  attention  to  detail  (ver¬ 
bal  identification  of  targets)  during 
target  sighting  through  a  simulated 
weapon  scope.  Demands  of  this  test 
task  are  consistent  with  rapidly 
changing  physical,  sensory,  and  cog¬ 
nitive  demands  in  a  combat 
environment. 

Conclusion 

Determination  about  service  mem¬ 
bers’  readiness  to  RTD  following 
mTBI  is  still  informed  primarily  by  a 
patient’s  self-report  of  symptoms  and 
by  clinical  tests  that  assess  perfor¬ 
mance  within  distinct  body  structure 
or  function  domains.  Widespread 
adoption  of  a  theoretical  framework 
that  measures  service  member  fit¬ 
ness  for  duty  at  the  activity  or  partic¬ 
ipation  level  would  be  highly  desir¬ 
able  to  improve  prognostication  of 
real-world  warfighting  performance. 
General  acceptance  of  a  paradigm 
that  conceives  of  an  individual’s 
readiness ,  not  as  the  absence  of 
impairments  but  as  a  vector-sum  of 
military  competencies,  represents  an 
important  ideological  shift  from 
what  a  member  cannot  do,  to  what 
he  or  she  can  do.  Although  this  type 
of  standards-based  construct  may  be 
difficult  to  quantify  using  conven¬ 


tional  impairment-based  testing, 
complex  assessment  methods 
should  help  to  bridge  this  assess¬ 
ment  gap. 

Measures  of  postconcussive  func¬ 
tional  performance  emerging  to 
address  RTD  assessment  challenges 
within  the  DoD  include  immersive 
virtual  environments;  field  or 
scenario-based  programs;  and  clini¬ 
cal  tests  incorporating  dual-task  and 
multitask  methods.  Although  each  of 
these  approaches  has  relative 
strengths  and  limitations,  all  are  chal¬ 
lenged  by  a  general  lack  of  clarity  on 
how  to  externally  validate  duty  read¬ 
iness  following  mTBI.  Absence  of  a 
“gold  standard”  benchmark  of  duty 
readiness  within  the  DoD  persists  as 
much  due  to  the  complexity  of  fac¬ 
tors  that  affect  human  performance 
following  neurotrauma  as  to  uncer¬ 
tainty  surrounding  how  to  measure 
such  a  multifaceted  construct.  Mea¬ 
surement  may  be  further  con¬ 
founded  by  the  expense  required  to 
install,  administer,  and  sustain  tech¬ 
nologically  sophisticated  or  inten¬ 
sive  assessment  programs,  dramati¬ 
cally  limiting  use  of  certain  methods 
outside  of  hub  military  treatment 
facilities.  Such  barriers  constrain  the 
widespread  feasibility  of  these 
approaches  and  make  DoD-wide 
standardization  of  RTD  metrics  diffi¬ 
cult.  Development  of  militarized 
dual-task  and  multitask  methods  rep¬ 
resent  a  potential  solution  to  these 
practice  and  dissemination  barriers 
given  the  relative  feasibility  of  clini¬ 
cal  assessment  techniques,  demon¬ 
strated  utility  of  dual-task  and  multi¬ 
task  assessment  in  civilian  patients 
with  TBI,  and  their  strong  face  valid¬ 
ity  for  commanders,  service  mem¬ 
bers,  and  clinicians.65’67’70’73"75’79’80 

Dual-task  and  multitask  testing  meth¬ 
ods  may  be  more  time  consuming  to 
administer  than  impairment-based 
assessments  and  not  necessarily  fea¬ 
sible  for  all  environments  of  care  82 
Nonetheless,  their  potential  sensitiv- 
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ity  to  duty-limiting  performance  gaps 
could  be  quite  valuable  in  remote 
clinical  practice  settings  where 
timely  and  appropriate  RTD  determi¬ 
nations  often  are  essential. 

Future  research  efforts  should  con¬ 
tinue  to  explore  and  develop 
standards-based  criteria  to  guide 
RTD  and  RTP  decision  making,  not 
only  in  the  wake  of  mTBI  but  also  to 
address  the  broad  spectrum  of 
potential  duty-  or  play-limiting  defi¬ 
cits.  Standards-based  metrics  do  not 
replace  traditional  clinical  decision 
making  by  clinicians  who  manage 
patients  and  their  injuries.  Such 
methods  provide  military  clinicians 
with  additional  data  points  for  eval¬ 
uating  abilities  more  clearly  related 
to  functional  occupational  demands. 
This  approach  ultimately  benefits 
the  service  member,  the  unit,  and 
the  military  as  a  whole  by  verifying 
that  a  returning  service  member  is 
not  only  symptom-free  but  truly 
“duty  ready.” 
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Development  and  Preliminary  Reliability  of  a  Multitasking 
Assessment  for  Executive  Functioning  After  Concussion 

Laurel  B.  Smith,  Mary  Vining  Radomski,  Leslie  Freeman  Davidson, 
Marsha  Finkelstein,  Margaret  M.  Weightman,  Karen  L.  McCulloch, 
Matthew  R.  Scherer 


OBJECTIVES.  Executive  functioning  deficits  may  result  from  concussion.  The  Charge  of  Quarters  (CQ)  Duty 
Task  is  a  multitask  assessment  designed  to  assess  executive  functioning  in  servicemembers  after  concussion. 
In  this  article,  we  discuss  the  rationale  and  process  used  in  the  development  of  the  CQ  Duty  Task  and  present 
pilot  data  from  the  preliminary  evaluation  of  interrater  reliability  (IRR). 

METHOD.  Three  evaluators  observed  as  12  healthy  participants  performed  the  CQ  Duty  Task  and  measured 
performance  using  various  metrics.  Intraclass  correlation  coefficient  (ICC)  quantified  IRR. 

RESULTS.  The  ICC  for  task  completion  was  .94.  ICCs  for  other  assessment  metrics  were  variable. 

CONCLUSION.  Preliminary  IRR  data  for  the  CQ  Duty  Task  are  encouraging,  but  further  investigation  is  needed 
to  improve  IRR  in  some  domains.  Lessons  learned  in  the  development  of  the  CQ  Duty  Task  could  benefit  future 
test  development  efforts  with  populations  other  than  the  military. 
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Concussion  has  received  unprecedented 
attention  in  the  military  because  of  the 
increased  incidence  in  the  past  decade 
(Helmick,  Baugh,  Lattimore,  &  Goldman, 
2012)  and  has  been  called  the  “signature 
injury”  of  the  conflicts  in  Iraq  and  Afghan¬ 
istan  (McCrea  et  ah,  2009,  p.  1369). 
Concussion  may  result  in  symptoms 
including  headache,  dizziness,  nausea,  sen¬ 
sitivity  to  noise  and  light,  slowed  thinking 
and  reaction  time,  memory  problems, 
difficulty  concentrating,  executive  dys¬ 
function,  and  visual  and  balance  changes 
(Carroll  et  ah,  2004).  Although  subtle 
and  sometimes  difficult  to  detect,  these 
multisensory  symptoms  can  negatively 
affect  job  performance  and  safety  in 
servicemembers. 

Army  occupational  therapists  play  key 
roles  in  evaluating  servicemembers  and 
making  recommendations  regarding  their 
ability  to  return  to  duty  after  concussion. 
Currently,  occupational  therapy  practi¬ 
tioners  rely  on  self-reported  symptoms  and 


vestibular  and  neuropsychological  as¬ 
sessments  to  determine  duty  readiness. 
However,  subjective  symptom  report  does 
not  always  coincide  with  clinical  recovery 
(Vagnozzi  et  al.,  2008),  and  neuropsy¬ 
chological  assessment  batteries  do  not  always 
predict  real-world  functioning,  especially 
after  a  combat  experience  (Brenner  et  al., 
2010).  Accurate  assessment  is  further  lim¬ 
ited  by  measures  with  ceiling  effects  or 
minimal  sensitivity  to  concussion-related 
deficits. 

Multitask  assessments  may  be  more 
sensitive  to  subtle  performance  deficits 
because  they  replicate  the  simultaneous 
cognitive  and  sensorimotor  demands  of 
unstructured,  complex  real-world  activ¬ 
ities  (Frisch,  Forstl,  Legler,  Schope,  & 
Goebel,  2012).  Despite  the  potential 
benefit  of  this  assessment  approach  and 
alignment  with  priorities  for  occupational 
therapy  evaluation,  few  options  exist  that 
have  satisfactory  reliability,  validity,  and 
clinical  utility  (Dawson  et  al.,  2009).  The 
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Multiple  Errands  Test  (MET;  Shallice  & 
Burgess,  1991)  is  an  example  of  a  multi¬ 
task  assessment  of  executive  functioning 
based  on  five  demands  of  multitasking: 
(1)  performing  multiple  but  discrete  tasks 
that  vary  in  priority,  complexity,  and 
length;  (2)  managing  interleaving  and 
dovetailing  tasks;  (3)  performing  tasks 
without  feedback;  (4)  dealing  with  inter¬ 
ruptions,  reprioritization,  and  rule  changes; 
and  (3)  self-initiating  task  changes  within 
the  activity  (Burgess,  2000).  The  many 
versions  of  the  MET  involve  completing  at 
least  10  unrelated  tasks  while  complying 
with  a  series  of  rules  in  either  a  shopping 
mall  or  hospital  lobby  setting  (Alderman, 
Burgess,  Knight,  &  Henman,  2003;  Cuberos- 
Urbano  et  al.,  2013;  Dawson  et  al.,  2009; 
Morrison  et  al.,  2013).  Although  the  MET 
appears  to  assess  “the  central  aspects  of 
executive  functioning  in  everyday  life” 
(Frisch  et  al.,  2012,  p.  257),  it  has  yet  to 
be  widely  adopted  in  clinical  practice  be¬ 
cause  of  site-specific  validation  require¬ 
ments,  time-intensive  administration,  and 
a  lack  of  standardized  scoring  manuals 
specific  to  each  site  (Radomski  &  Morrison, 
2014). 

A  team  of  military  and  civilian  oc¬ 
cupational  and  physical  therapists  are 
currently  developing  a  performance-based 
assessment  battery  called  the  Assessment 
of  Military  Multitasking  Performance 
(AMMP;  Radomski  et  al.,  2013).  The 
AMMP  includes  six  dual-  and  multitask 
assessments  designed  to  assess  concussion- 
related  deficits.  If  proven  reliable  and  valid, 
the  AMMP  will  be  used  by  military  occu¬ 
pational  and  physical  therapists  to  determine 
duty  readiness  for  servicemembers  after 
concussion. 

The  Charge  of  Quarters  (CQ)  Duty 
Task  (CQDT)  was  developed  as  one  of  the 
assessments  included  in  the  AMMP  bat¬ 
tery  that  uses  the  structure  of  the  MET  to 
assess  executive  functioning.  CQ  duty  is 
an  additional  duty  in  the  military  during 
which  servicemembers  are  responsible  for 
24-hr  supervision  and  security  of  a  facility; 
servicemembers  on  CQ  duty  are  fre¬ 
quently  tasked  with  various  assignments 
that  are  unstructured  and  unrelated  in 
nature.  This  scenario  provides  a  realistic 
backdrop  for  the  multitask  assessment 
given  the  reality  of  task  demands  and 


face  validity  among  servicemembers.  This 
article  describes  the  rationale  and  develop¬ 
ment  process  of  the  CQDT  and  presents 
pilot  data  from  the  preliminary  evaluation 
of  interrater  reliability  (IRR). 

Description  of  the  Charge  of 
Quarters  Duty  Task 

In  the  CQDT,  as  in  the  MET,  participants 
receive  in-depth  instructions  and  a  written 
list  of  assignments  and  performance  rules. 
They  are  required  to  visit  four  different 
hypothetical  work  areas  (marked  with 
duct  tape):  (1)  the  CQdesk,  (2)  the  bulletin 
board,  (3)  the  supply  closet,  and  (4)  the 
assembly  area,  each  containing  the  in¬ 
formation  and  resources  necessary  to  com¬ 
plete  their  assignments.  They  are  encouraged 
to  keep  transits  between  work  areas  to 
a  minimum  (seven  or  fewer)  and  are  told 
to  revisit  an  area  only  if  necessary  to  com¬ 
plete  the  task.  Task  assignments  include 
reporting  a  CQ  duty  shift  change,  assem¬ 
bling  a  footstool  from  PVC  pipe,  reporting 
the  number  of  vacant  rooms  in  the  barracks 
(living  quarters  for  servicemembers)  using 
a  barracks  layout,  conducting  an  inventory 
of  PVC  supplies,  obtaining  the  address  of 
the  manufacturer  of  the  footstool  mate¬ 
rials,  locating  the  telephone  number  of 
another  servicemember  using  a  personnel 
roster,  and  locating  the  room  of  a  specified 
servicemember  using  a  map  of  a  barracks 
layout. 

During  the  exercise,  participants  must 
adhere  to  four  rules:  (1)  Assemble  the 
footrest  only  in  the  assembly  area,  (2)  bring 
only  the  number  of  PVC  parts  needed  for 
the  footrest  to  the  assembly  area,  (3)  do  not 
move  or  remove  any  of  the  materials  from 
the  walls  in  any  of  the  work  areas,  and  (4) 
do  not  speak  to  the  examiners  during  the 
assessment.  Throughout  the  task,  partic¬ 
ipants  must  also  deal  with  interruptions  and 
reprioritization  of  tasks.  Scoring  metrics 
borrowed  from  the  MET  include  accuracy 
of  task  performance  (Cuberos-Urbano  et  al., 
2013;  Dawson  et  al.,  2009;  Morrison  et  al., 
2013),  total  rule  breaks  (Cuberos-Urbano 
et  al.,  2013;  Dawson  et  al.,  2009;  Morrison 
et  al.,  2013),  frequency  of  rule  breaks 
(Dawson  et  al.,  2009),  transits  between  work 
areas  (Morrison  et  al.,  2013),  and  total 
performance  time. 


440 

Downloaded  From:  http://ajot.aota.org/  on  07/09/2014  Terms  of  Use:  http://AOTA.org/terms 


Method 

Instrument  Development 

The  CQDT  was  developed  as  part  of  the 
AMMP  battery.  The  initial  version  of  the 
AMMP  included  five  multitask  assessments 
and  three  dual-task  assessments  (Radomski 
et  al.,  2013).  After  initial  pilot  testing  of 
the  AMMP  battery,  data  analysis  indicated 
variable  IRR  (intraclass  correlation  coef¬ 
ficients  [ICCs]  of  .45,  .37,  and  .79  for  task 
performance)  for  the  three  multitask  as¬ 
sessments  of  executive  functioning.  Scoring 
was  complicated  by  errors  resulting  from 
simultaneous  observation  and  scoring  re¬ 
quirements  and  by  a  lack  of  clearly  defined 
scoring  criteria  outlining  acceptable  toler¬ 
ances  for  partially  accurate  task  perfor¬ 
mance.  For  example,  when  participants 
were  told  to  obtain  an  address,  rater  dis¬ 
agreements  occurred  if  part  of  the  address 
was  incorrect  (e.g.,  transposed  digits,  spell¬ 
ing  errors);  some  examiners  gave  full  credit 
for  task  completion  and  others  gave  no 
credit.  In  addition  to  multiple  scoring 
challenges,  test  developers  indicated  sub¬ 
stantial  test  burden  from  three  relatively 
similar  multitask  assessments  and  limited 
face  validity  of  the  tasks  as  reported  by 
participants.  In  an  effort  to  improve  IRR, 
face  validity,  and  clinical  feasibility,  the 
CQDT  was  developed  to  replace  the  three 
previous  iterations  of  multitask  assessments. 

The  first  step  in  the  development  of 
the  CQDT  was  to  reexamine  the  literature 
pertaining  to  current  multitask  assess¬ 
ments.  The  team  also  shared  the  initial 
concept,  materials,  and  instructions  of  the 
CQDT  with  a  panel  of  experienced  service- 
members  who  provided  recommendations 
to  improve  face  validity  of  the  task  with  the 
target  population.  On  the  basis  of  the 
definition  of  multitasking  (Burgess,  2000) 
and  feedback  from  subject  matter  experts, 
the  team  created  a  list  of  parameters  to  be 
tested. 

Once  the  initial  task  was  developed, 
test  developers  practiced  administering 
the  task  on  servicemembers  and  civilians 
to  observe  variations  in  performance  and 
variations  in  the  interpretation  of  perfor¬ 
mance  by  multiple  evaluators.  After  practice 
administrations,  test  developers  clarified 
task  instructions  and  revised  the  approach 
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to  scoring  by  creating  operational  definitions 
that  clarified  situations  in  which  no  credit, 
partial  credit,  or  full  credit  should  be  given. 
These  operational  definitions  were  included 
on  the  score  sheet.  For  example,  a  par¬ 
ticipant  who  reported  the  incorrect 
number  of  barracks  rooms  would  receive 
partial  credit  for  task  performance  in  that 
domain  as  determined  by  the  operational 
definition  for  that  task.  This  scoring  ap¬ 
proach  reduced  scoring  complexity  and 
allowed  raters  to  assign  a  score  quickly  upon 
observation  of  task  completion. 

The  score  sheet  was  also  improved  to 
reduce  scoring  errors  resulting  from  simul¬ 
taneous  observation  and  scoring  require¬ 
ments.  Many  aspects  of  the  CQDT  required 
scoring  in  real  time  (i.e.,  radio  communi¬ 
cations  with  various  personnel  on  the  correct 
radio  frequency)  to  determine  whether 
participants  completed  tasks  independently 
and  accurately  or  required  cueing.  Raters 
who  were  distracted  or  who  failed  to  score 
performance  on  these  tasks  immediately 
made  scoring  errors.  To  address  this  issue, 
task  assignments  were  listed  chronologically 
on  the  score  sheet,  and  tasks  requiring  im¬ 
mediate  scoring  were  emphasized  with  bold 
font.  This  design  helped  cue  the  evaluators 
to  ensure  observation  of  performance  at  ap¬ 
propriate  times.  Last,  the  score  sheet  included 
correct  responses  for  objective  performance 
components  (e.g.,  correct  number  of  vacant 
barracks  rooms  to  be  reported,  manufacturers 
address),  allowing  the  rater  to  quickly  identify 
performance  accuracy  and  assign  the  ap¬ 
propriate  score.  These  additions  were  im¬ 
plemented  to  maximize  scoring  efficiency. 

After  all  modifications  were  made  to 
the  CQDT,  test  developers  piloted  the 
revised  multitask  assessment  in  a  healthy 
population  to  assess  IRR.  Given  the  an¬ 
ticipated  variability  in  task  performance 
between  healthy  servicemembers  and  those 
with  concussion,  evaluation  of  IRR  in 
healthy  servicemembers  allowed  for  sub¬ 
sequent  scoring  and  procedural  refine¬ 
ments  to  be  made  before  evaluating  IRR  in 
servicemembers  with  concussion. 

Intrarater  Reliability  Testing 

Preliminary  IRR  was  assessed  between  3 
(2  trained  and  1  novice)  raters  when  mea¬ 
suring  individual  participant  performance 


on  the  CQDT.  The  two  trained  raters 
were  involved  in  test  development,  and  the 
novice  rater  was  a  physical  therapist  with 
no  prior  experience  with  the  CQDT.  This 
design  helped  determine  whether  inex¬ 
perienced  providers  could  easily  and  accu¬ 
rately  score  the  assessment.  Before  evaluating 
participants,  the  novice  rater  received  a 
brief  orientation  (<30  min)  to  the  score  sheet, 
performance  metrics,  and  operational 
definitions  of  task  performance,  rules,  and 
rule  breaks.  IRR  was  established  for  all 
raters. 

Participants 

Participants  were  recruited  by  convenience 
sampling  from  the  U.S.  Army  Research 
Institute  of  Environmental  Medicine  in 
Natick,  Massachusetts.  All  healthy  active- 
duty  servicemembers  (active  duty,  guard, 
or  reserve  component)  ages  1 8-42  yr  were 
eligible  to  participate.  Participants  were 
excluded  if  they  reported  a  history  of 
traumatic  brain  injury  (TBI)  or  concussion 
in  the  previous  year,  any  documented 
active-duty  restrictions  (currently  on  a  mili¬ 
tary  profile),  any  physical  or  behavioral 
health  condition  preventing  sustained  ac¬ 
tivity  for  up  to  30  min,  history  of  psychiatric 
disorder,  and  uncorrected  hearing  deficits. 
All  participants  gave  written  informed 
consent  before  participation,  and  the  in¬ 
stitutional  review  board  at  the  U.S.  Army 
Research  Institute  of  Environmental  Med¬ 
icine  approved  the  study. 

Data  Collection 

The  following  components  were  measured 
via  observation: 

•  Task  completion  was  defined  as  the  extent 
to  which  participants  independently  and 
accurately  completed  each  assignment. 
Each  assignment  was  scored  0  ( not  com¬ 
plete) ,  1  ( partially  complete  or  required 
cueing  to  complete ),  or  2  ( completed  to  de¬ 
fined  standard  independently  without  cue¬ 
ing).  The  test  included  17  assignments 
(some  assignments  required  more  than 
one  task),  with  up  to  2  points  possible  for 
each,  for  a  total  of  34  possible  points  for 
task  completion. 

•  Total  rule  breaks  for  the  four  rules  were 
operationally  defined  on  the  score  sheet. 
Each  rule  that  was  broken  was  recorded. 


•  Frequency  of  rule  breaks  was  recorded 
for  each  rule;  it  was  possible  to  break 
the  same  rule  multiple  times.  No  limit 
was  placed  on  the  frequency  of  rule 
breaks. 

•  Performance  time  was  defined  as  the  to¬ 
tal  time  to  complete  the  task. 

•  Transits  were  defined  as  movements  be¬ 
tween  work  areas.  Leaving  one  work 
area  and  entering  another  was  consid¬ 
ered  one  transit. 

Data  Analysis 

The  ICC  was  used  to  quantify  preliminary  IRR 
The  Krippendorff  (Hayes  &  Krippendorff, 
2007)  a  macro  was  run  under  SPSS 
Version  18.0  (IBM  Corporation,  Ar- 
monk,  NY)  to  generate  the  ICCs.  Twelve 
cases  provided  93%  confidence  to  mea¬ 
sure  our  objective  for  an  ICC  of  .90 
against  a  minimum  ICC  of  .70  (Bonett, 
2002).  For  metrics  that  achieved  an  ICC 
of  .90,  the  mean,  standard  deviation,  and 
range  are  reported  on  the  basis  of  the  median 
of  the  three  scores  for  each  participant. 

Results 

A  total  of  12  servicemembers  (7  men  and  5 
women)  participated  in  this  study.  The 
mean  time  to  perform  the  CQDT  was 
19.6  min;  7  of  12  participants  completed 
the  task  in  <20  min  and  11  of  12  in  <23 
min.  The  maximum  test  duration  was  31.9 
min.  The  average  number  of  transits  was 
10.3.  Table  1  provides  the  IRR  results. 
Rule  breaks  and  frequency  of  rule  breaks 
were  not  reliable,  with  ICCs  of  .66  and  .64, 
respectively.  Task  completion,  transits,  and 
total  time  were  highly  reliable,  with  ICCs 
of  .94,  .98,  and  .98,  respectively. 

Discussion 

Occupational  therapists  are  charged  with 
developing  and  implementing  measurement 
strategies  that  characterize  the  extent  to  which 
impairments  impede  daily  life  performance 
(Baum,  Perlmutter,  &  Dunn,  2005).  Doing 
so  is  difficult  when  impairments  such  as  ex¬ 
ecutive  dysfunction  are  potentially  difficult  to 
detect,  as  in  servicemembers  with  concus¬ 
sion.  Performance-based  assessments  that 
involve  multitasking  have  demonstrated 
the  potential  to  discriminate  between 
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Table  1.  Preliminary  Interrater  Reliability  Results  for  the  Charge  of  Quarters  Duty  Task 
(N=  12) 


Item 

Reliability  (ICC) 

95%  Cl 

Mean  (SD) 

Range 

Task  completion 

.94 

[.86,  .99] 

27.6  (5.6) 

13-33 

Rule  breaks 

.66a 

[.39,  .88] 

Frequency  of  rule  breaks 

.64b 

[.32,  .90] 

Transits 

.98 

[.96,  .99] 

10.5  (4.0) 

5-18 

Total  time  (min) 

.98 

[.96,  .99] 

19.6  (4.8) 

13.2-31.9 

Note.  Cl  =  confidence  interval;  ICC  =  intraclass  correlation  coefficient;  SD=  standard  deviation.  The  mean, 
standard  deviation,  and  range  are  reported  only  for  metrics  that  achieved  an  ICC  of  .90. 
aFour  of  1 2  triplets  did  not  agree.  bSix  of  1 2  triplets  did  not  agree. 


healthy  control  participants  and  people 
with  executive  dysfunction  (Alderman  et  al., 
2003;  Baum  et  al.,  2008;  Morrison  et  al., 
2013;  Wolf,  Morrison,  &  Matheson,  2008) 
and  may  be  an  alternative  to  traditional 
measures  of  cognitive  domains,  which  often 
fail  to  detect  existing  deficiencies  in  complex 
task  performance  (Tranel,  Hathaway-Nepple, 
&  Anderson,  2007).  Although  such  tests  do 
not  appear  to  be  subject  to  the  ceiling  effects 
of  more  structured  measures  of  performance 
(Hall  et  al.,  1996;  Scott  et  al.,  201 1),  they  are 
typically  complex  to  administer  and  score 
(Morrison  et  al.,  2013).  More  multitasking 
tests  that  are  specific  to  various  clinical  pop¬ 
ulations  and  life  situations  are  needed.  IRR 
specific  to  servicemembers  with  concussion 
and  discriminant  validity  remain  untested  for 
the  CQDT,  but  the  preliminary  evaluation  of 
IRR  in  healthy  participants  suggests  progress 
in  the  development  of  a  multitask  assessment 
of  executive  functioning  for  servicemembers 
with  concussion. 

The  current  evaluation  of  preliminary 
IRR  highlights  easily  scored  metrics  for 
multitasking  assessment  and  those  requiring 
further  refinement  by  the  research  team.  IRR 
for  task  completion  improved  from  previous 
versions  of  multitasking  assessments  because 
the  score  sheet  was  redesigned  to  include 
operational  definitions  and  list  performance 
tasks  chronologically.  These  elements  helped 
clarify  scoring  criteria  and  reduce  rater  dis¬ 
agreements  regarding  task  performance. 

Unfortunately,  behavioral  aspects  of  rule 
breaks  and  frequency  of  rule  breaks  were  not 
as  well  specified,  accounting  for  continued  but 
solvable  problems  with  IRR.  Rater  disagree¬ 
ments  in  how  to  score  vocalizations  directed  at 
the  examiners  (e.g.,  asking  the  examiner 
questions)  and  the  number  of  PVC  parts 
brought  to  the  assembly  area  largely  ex¬ 
plained  the  unacceptable  ICCs  for  rule 


breaks  and  frequency  of  rule  breaks.  Op¬ 
erational  definitions  were  not  clear  enough 
to  account  for  the  unpredictable  nature  of 
human  performance  in  these  areas.  Addi¬ 
tionally,  the  restricted  range  resulting  from 
only  four  rules  may  have  had  a  negative 
impact  on  the  ICC  values.  With  a  restricted 
range,  one  missed  observation  in  rule  breaks 
can  affect  the  ICC  value  to  a  greater  degree 
than  with  a  greater  number  of  rules.  In 
preparation  for  future  data  collection,  op¬ 
erational  definitions  have  been  revised  and 
piloted  to  improve  IRR  for  rule  breaks. 

Limitations  and  Future  Directions 

The  CQDT  is  in  relative  infancy  in  terms  of 
test  development.  Thus  far,  clinical  feasi¬ 
bility  and  IRR  for  the  CQDT  have  been 
evaluated  in  only  a  small  number  of  healthy 
participants.  Results  of  future  data  collection 
will  determine  IRR  and  clinical  feasibility  of 
the  CQDT  in  a  clinical  population  and,  most 
important,  will  ascertain  whether  it  discrim¬ 
inates  between  healthy  control  participants 
and  servicemembers  with  concussion.  If  so, 
further  research  will  need  to  be  conducted  to 
determine  whether  the  CQDT  predicts  suc¬ 
cessful  return  to  duty.  Finally,  the  team  is 
exploring  the  development  of  a  civilian  version 
of  the  CQDT  that  could  be  used  as  a  stand¬ 
alone  assessment  of  executive  dysfunction. 

Implications  for  Occupational 
Therapy  Practice  and  Research 

The  results  of  this  study  have  the  following 
implications  for  occupational  therapy  prac¬ 
tice  and  research: 

•  Performance-based  assessments  of  mul¬ 
titasking  may  enable  occupational  ther¬ 
apy  practitioners  to  identify  executive 
function  deficits  after  concussion. 
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•  Because  of  the  complexity  of  scoring 
a  multitask  assessment,  operational  defini¬ 
tions  for  scoring  are  best  developed  on  the 
basis  of  observed  variations  in  task  perfor¬ 
mance  and  differences  in  interpretation  of 
that  performance  by  multiple  evaluators. 

•  The  lessons  learned  in  the  development 
of  the  CQDT  may  benefit  occupational 
therapy  practitioners  interested  in  devel¬ 
oping  performance-based  assessments  of 
executive  dysfunction  tailored  to  popu¬ 
lations  and  practice  settings  other  than 
the  military. 

Conclusion 

There  remains  a  need  for  reliable,  valid, 
and  clinically  feasible  assessments  that  can 
be  used  to  identify  executive  dysfunction. 
Performance-based  assessments  that  in¬ 
corporate  multitask  methods  and  accu¬ 
rately  simulate  job  demands  may  prove 
useful  for  occupational  therapy  practi¬ 
tioners  in  determining  return-to-activity 
timelines  in  various  populations.  A 
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INTRODUCTION 

Combat-related  exposures,  routine  operational  and  training  activities  as  well  as  common  sports 
and  leisure  activities  all  put  military  service  members  (SM)  at  increased  risk  for  sustaining  a 
traumatic  brain  injury  (TBI)  in  both  theater  and  garrison  environments.  In  excess  of  320,000 
Service  Members  (SM)  sustained  at  least  one  TBI  from  2000  through  the  end  of  2014  with  more 
than  82%  of  these  injuries  classified  as  mild  (http://dvbic.dcoe.mil/dod-worldwide-numbers-tbi, 
accessed  lMay2015).  Military  personnel  who  have  been  concussed  may  experience  persistent 
cognitive,  postural  and  dynamic  stability  deficits  in  addition  to  disabling  headaches,  sleep 
dysfunction,  auditory,  vestibular,  and  visual  impairments.  The  occurrence  of  such  symptoms 
and  deficits  can  limit  safe  and  effective  job  performance  in  the  inherently  demanding  military 
profession  [1}.  While  most  personnel  recover  within  days  or  weeks  after  concussion,  for  some 
individuals  [2],  symptoms  can  persist  impairing  duty  performance  and  disrupting  a  SM's  ability 
to  safely  return  to  duty. 

Rehabilitation  clinicians  currently  use  a  variety  of  metrics  to  evaluate  acute  concussion 
including  subjective  symptom  reports  combined  with  neuropsychological  testing  and 
standardized  postural  stability  assessments  such  as  the  Balance  Error  Scoring  System  [3].  These 
approaches,  based  on  a  sports  medicine  model,  typically  involve  comparison  of  post-injury 
results  to  a  pre-injury  baseline  test  of  the  same  measures.  Pre-injury  baseline  tests  are  typically 
not  available  for  many  in  the  military,  so  alternatives  are  needed.  Presently,  there  are  limited 
options  for  military-based  functional  assessments  to  evaluate  SM  following  concussive  injury 
[4].  In  deployed  settings,  Department  of  Defense  policy  mandates  the  use  of  functional 
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assessment  measures  for  Soldiers  after  three  or  more  concussive  events  to  guide  return  to  duty 
(RTD)  decision-making  in  that  environment  [1].  Given  the  inherent  risks  associated  with 
premature  return  to  combat  [4],  this  approach  acknowledges  the  importance  of  assessing 
complex  task  performance  under  realistic  conditions.  It  is  noteworthy  however,  that  specific 
guidance  on  the  type,  difficulty  and  duration  of  such  assessments  has  not  been  established, 
defined  or  standardized  by  the  DoD  or  other  potential  stakeholders  in  the  professional  athletics 
community  [1,4].  To  address  these  gap  areas  in  functional  performance  testing  for  military 
personnel,  our  military-civilian  rehabilitation  research  team  developed  a  post  concussive 
functional  assessment  battery  to  provide  RTD  guidance  following  mild  traumatic  brain  injury  [4]. 
The  Assessment  of  Military  Multitasking  Performance  (AMMP)  is  a  battery  of  functional  dual¬ 
tasks  and  multitasks  that  simulate  the  combined  sensorimotor,  cognitive,  and  exertional 
demands  of  common  Soldier  tasks.  The  initial  AMMP  development  which  was  informed  by 
military  stakeholder  inquiry,  expert  consultation  &  literature  review,  has  been  described 
elsewhere  [5].  In  brief,  the  AMMP  test  battery  initially  consisted  of  nine  dual-task  and  multitask 
assessments  which  were  developed  by  members  of  the  research  team  with  one  investigator 
serving  as  the  principal  developer  of  each  task.  Task  were  designed  to  challenge  distinct  mTBI 
related  vulnerabilities  with  some  redundancy  in  terms  of  the  domains  or  vulnerabilities  tested. 

The  AMMP  purports  to  measure  a  complex  and  heterogeneous  concept:  "readiness  for  duty". 
Unlike  similar  functional  performance  measures  however,  "readiness  to  RTD"  is  not  a  construct 
that  resides  in  an  isolated  performance  domain  such  as  physical  capability  [3,6-11]  or  executive 


function  [12,13].  Common  Soldier  tasks  require  decision-making,  intact  cognitive  and  sensory 
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function,  often  in  dynamic  and  stressful  environments  requiring  elite  physical  abilities.  As  such, 
valid  measurement  of  integrated  functional  performance  can  be  a  challenging  process  relative 
to  standard  clinical  assessment  of  isolated  performance  in  physical,  sensory  or  cognitive 
domains.  The  AMMP  team  focused  on  development  of  a  performance  based  assessment  that 
targeted  known  mTBI  vulnerabilities.  The  AMMP  had  to  meet  ecological  validity  standards  of 
military  personnel,  including  commanders,  who  value  the  real-world  use  of  the  assessment  [5]. 
In  addition,  practical  considerations  including  the  administration  time,  test  space  requirements, 
and  cost  and  durability  of  test  materials  were  all  factors  that  required  consideration  in  the 
development  of  the  AMMP  [5,10].  Given  the  consequences  of  using  AMMP  metrics  to 
contribute  to  duty  readiness  decisions,  a  vital  first  step  in  the  AMMP  validation  process  was  to 
determine  if  the  AMMP  tasks  could  be  scored  reliably. 

The  purpose  of  this  article  is  to  describe  the  iterative  task  refinement  process  for  the 
Assessment  of  Military  Multitasking  Performance  including  feasibility  considerations  and  the 
use  of  repeated  inter-rater  reliability  testing  to  improve  task  metrics,  administration  and 
scoring  instructions. 


METHODS 

This  measurement  development  study  involved  three  sequential  phases  which  used  inter-rater 
reliability  findings,  informal  and  qualitative  feedback  from  raters  and  subjects,  as  well  as 
logistical  evaluation  by  the  test  developers  of  practical  properties  to  drive  task  revision. 
Specifically,  the  team  integrated  lessons  learned  during  testing  in  an  iterative  manner  over 
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successive  phases  of  testing  to  improve  face  validity  of  the  tasks  and  the  quality  of  subject 
performance  data.  Investigators  also  worked  to  decrease  test  burden  on  participants  by 
decreasing  test  administration  time.  Other  considerations  during  task  development  and 
refinement  included  consideration  of  the  cost  and  durability  of  equipment,  and  testing  space 
requirements.  Revisions  included  refinement  of  task  administration  and  scoring  instructions, 
operational  definitions  and  scoring  metrics  for  the  AMMP  tasks.  All  subjects  provided  informed 
consent  as  specified  by  the  relevant  institutional  review  board  prior  to  testing  at  each  phase  of 
AMMP  development.  Specifically  study  protocols  were  approved  by  the  IRB  at  the  United  States 
Army  Research  Institute  of  Environmental  Medicine  (USARIEM)  and  the  Womack  Army  Medical 
Center  (WAMC)  at  Fort  Bragg. 

Phase  I:  Testing  in  active  duty  healthy  control  Soldiers 

The  goal  of  this  Phase  I  was  to  evaluate  the  inter-rater  reliability  (IRR)  and  feasibility  of  the 
AMMP  when  administered  to  military  healthy  control  (HC)  subjects.  There  were  2 
subcomponents  to  this  phase.  During  Phase  la,  we  evaluated  the  feasibility  and  IRR  of  the 
original  9  tasks  comprising  the  original  version  of  the  AMMP  [4,5].  Three  of  the  original 14  5 
multitasks  were  eliminated  and  salient  components  were  refined  and  consolidated  into  one 
multitask  called  the  Charge  of  Quarters  (CQ)  Duty  Task  (see  Figure).  The  3  original  multitasks 
demonstrated  poor  IRR  due  to  unclear  operational  definitions  of  success  and  failure  on  task 
components  (Figure),  and  difficulty  observing  all  components  of  each  task.  One  of  the  4 
original  dual-tasks,  the  Step  initiation-Stroop  task  was  dropped  after  this  phase  of  testing  in 
favor  of  dual  task  assessments  with  greater  face  validity  [4,5]  and  due  to  concerns  about 


durability  of  instrumentation.  For  Phase  Mb,  feasibility  and  IRR  of  2  test  tasks  were  evaluated, 
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the  new  multitask,  CQ  Duty  and  a  revised  SALUTE  multitask  was  modified  to  incorporate 
improvised  explosive  device  (IED)  marker  reporting.  In  a  later  phase  of  this  study,  the  SALUTE 
multitask  was  modified  to  incorporate  the  reaction  (Rx)  time  dimension  of  the  eliminated  Step 
initiation-Stroop  task. 

Participants. 

Healthy  subjects  between  the  ages  of  18  and  42  were  recruited  by  convenience  sampling  from 
both  human  research  volunteers  (HRV)  and  permanent  party  active  duty  service  members  from 
the  U.S.  Army  Research  Institute  of  Environmental  Medicine  (USARIEM)  in  Natick, 
Massachusetts.  Participants  were  excluded  if  they  reported  a  history  of  concussion  within  the 
previous  year  or  had  any  residual  symptoms  from  a  prior  concussion.  Subjects  were  also 
excluded  for  1)  documented  duty  limiting  profile  for  physical  or  behavioral  health  condition 
preventing  continuous  activity  for  up  to  30  minutes,  2)  history  of  psychiatric  disorder  and  3) 
uncorrected  hearing  or  visual  deficits  preventing  functional  hearing  or  vision.  In  Phase  la,  the 
initial  9  AMMP  tasks  were  evaluated  on  20  healthy  control  (HC)  volunteers  (11  males,  9 
females,  mean  age  25.8  (+/-3.5))  with  revisions  made  that  required  re-evaluation  of  IRR  on  2  of 
the  tasks.  In  Phase  lb,  these  two  revised  tasks  (CQ  Duty  and  SALUTE-Exertion  task)  were  tested 
on  12  USARIEM  subjects  who  were  a  subset  of  the  subjects  tested  in  Phase  la. 

Data  Collection 

In  Phase  la,  to  standardize  the  AMMP  task  administration  for  initial  IRR  evaluation,  raters 
reviewed  the  task  administration  instructions,  the  subject  instructions,  and  the  task  score 
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sheets  on  his/her  own  prior  to  the  beginning  of  testing.  The  principal  developer  for  each  task 
provided  a  brief  training  session  followed  by  a  practice  scoring  session  using  teams  of  3  raters 
with  a  mock  subject.  The  raters  then  discussed  scoring  discrepancies  and  came  to  consensus 
on  operational  definitions  for  scoring  as  necessary,  which  were  then  included  in  the  guidelines 
for  the  task.  If  necessary,  a  second  mock  subject  was  scored  with  additional  discussion  to  arrive 
at  a  clear  consensus  among  raters.  Each  scorer  individually  rated  20  active  duty  service 
member  volunteers  on  each  of  4  or  5  of  the  tasks.  The  scorers  were  both  physical  (PT)  and 
occupational  (OT)  therapists  (co-authors)  with  at  least  one  of  the  raters  considered  novice  or 
new  with  limited  knowledge  or  exposure  to  a  particular  task.  Novice  raters  had  not 
participated  in  the  development  of  the  task  or  pilot  tested  training  materials  for  a  given  task, 
thereby  providing  an  unbiased  assessment  of  their  experience  in  rating  the  task.  This  iterative 
approach  was  used  to  fine  tune  the  testing  materials  and  scoring  sheets  for  the  tasks  over  the 
course  of  testing  (Figure).  During  testing,  one  rater  read  the  scripted  instructions  and  interacted 
with  the  subject  and  all  3  raters  scored  the  subject  from  direct  observation.  Additionally,  test 
developers  provided  informal  feedback  throughout  the  reliability  testing  sessions  to  discuss 
issues  with  feasibility,  face  validity  and  general  practicality  of  tasks  and  ratings.  In  Phase  lb,  12 
HC  volunteers  from  USARIEM  were  tested  on  the  revised  CQ.  Duty  task  and  the  revised  SALUTE- 
exertion  task  by  a  team  of  2  OTs  and  1  PT. 

Phase  II:  Testing  in  active  duty  Soldiers  with  persistent  mTBI  related  symptoms 
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The  goal  of  Phase  II  was  to  evaluate  the  IRR  and  feasibility  of  the  revised  AMMP  tasks  outlined 
in  Table  1  in  subjects  with  mTBI  residual  symptoms.  This  phase  of  testing  took  place  at  Fort 
Bragg,  NC  with  subjects  primarily  in  military  occupation  specialties  related  to  combat  and 
combat  support. 

Participants 

A  total  of  12  SM  (11  male,  median  age  31,  range  21-42)  with  mTBI  were  recruited  from  the 
clinical  population  receiving  rehabilitation  services  at  the  Womack  Army  Medical  Center 
(WAMC)  TBI  Clinic  for  persistent  symptoms  from  a  concussion  occurring  from  2  weeks  to  2 
years  (median  days  (range):  306  (71-470))  prior  to  the  AMMP  test  date.  Physical  and 
occupational  therapists  from  the  WAMC  Bl  Clinic  identified  potential  participants  who  met  the 
eligibility  criteria  and  provided  an  information  and  study  contact  form.  Participants  were 
excluded  as  described  in  Phase  I.  A  second  set  of  7  SM  (6  male)  were  recruited  from  this  same 
population  for  additional  reliability  testing  focused  primarily  on  revisions  to  the  SALUTE- 
exertion  task  scoring  and  instructions  to  subjects. 

Data  Collection 

Prior  to  data  collection  at  Fort  Bragg,  raters  were  briefed  by  each  AMMP  task  developer  on 
changes  in  test  administration  or  scoring  that  resulted  from  prior  IRR  testing  on  HC  subjects.  A 
practice  subject  was  tested  by  all  raters  and  any  discrepancies  in  scoring  were  discussed  and 
clarified  by  the  rater  team  before  subject  testing.  Three  raters  were  present  for  this  data 


collection  phase  which  took  place  over  2  separate  test  weeks.  During  a  single  3  hour  session 


11 


subjects  were  consented,  provided  intake  information,  completed  neurocognitive  testing  and 
the  AMMP.  Some  subjects  were  not  tested  on  all  AMMP  tasks  for  various  reasons  including 
other  appointments,  duty  requirements  which  took  precedence  over  study  participation  or 
poor  tolerance  to  some  tasks.  The  order  of  AMMP  task  administration  varied  to  minimize  a  test 
order  effect.  Following  modifications  to  tasks  based  on  findings  from  these  subjects,  an 
additional  7  subjects  with  mTBI  (6  male,  1  female)  were  tested  using  a  different  team  of  3 
raters.  These  7  subjects  were  evaluated  on  all  AMMP  tasks,  however,  the  focus  of  the  IRR  was 
on  further  revisions  to  the  SALUTE  -exertion  task.  By  this  point,  most  scoring  or  administrative 
changes  for  5  of  the  6  test  tasks  were  minor  and  primarily  involved  modifications  to  operational 
definitions,  improvements  to  scoresheets  or  clarifications  of  instructions  to  subjects.  Following 
this  Phase  II,  significant  revisions  were  made  to  the  SALUTE-exertion  task  including  renaming  it 
the  PATROL-exertion  task  (Figure).  Our  original  intention  [5]  was  to  have  the  rater  teams 
blinded  to  the  subject  group  for  each  subject.  For  practical  reasons  including  recruitment 
difficulties,  subject  testing  could  not  be  done  in  larger  cohorts  that  would  have  allowed  raters 
to  be  blinded  to  group. 

Phase  III:  Testing  HC  active  duty  SM  and  SM  with  persistent  mTBI  related  symptoms 

The  goal  of  this  Phase  III  was  to  evaluate  the  inter-rater  reliability  (IRR)  and  feasibility  of  the 
AMMP  tasks  after  the  final  revisions  to  all  tasks  were  completed.  Both  HC  and  subjects  with 


mTBI  were  evaluated  in  this  phase. 
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Participants 

Healthy  control  subjects  were  recruited  by  convenience  sampling  of  volunteers  from  Fort  Bragg 
Special  Operations  and  the  528  Sustainment  Brigade,  subjects  who  responded  to  recruitment 
postings  placed  at  fitness  centers  and  cafeterias  around  the  base,  as  well  as  volunteers  from  the 
Defense  and  Veterans  Brain  Injury  Center  (DVBIC)  in-processing  briefings.  These  DVBIC 
briefings  occurred  on  an  almost  daily  basis  as  part  of  the  standard  informational  training 
provided  to  all  soldiers  in-processing  to  Fort  Bragg  after  transfer  from  other  duty  stations. 
Subjects  with  mTBI  were  recruited  from  the  clinical  population  receiving  rehabilitation  services 
at  the  WAMC  TBI  Clinic  using  inclusion  criteria  as  described  in  Phase  II  above.  A  total  of  26 
subjects  were  involved  in  this  final  phase  of  IRR  evaluation,  7  healthy  control  (5  male,  median 
(range)  age  34(20-42))  and  19  subjects  with  residual  mTBI  symptoms  (all  male,  median  (range) 
age  24  (19-40)).  Median  days  since  most  recent  concussion  was  147  (range  63-632). 

Data  Collection 

Changes  that  resulted  from  Phase  II  IRR  testing  were  reviewed  by  task  developers  with  the  rater 
team  to  standardize  AMMP  task  administration.  By  this  point,  most  scoring  or  administrative 
changes  for  5  of  the  6  test  tasks  were  minor  and  primarily  involved  modifications  to  operational 
definitions,  improvements  to  scoresheets  or  clarifications  of  instructions  to  subjects.  This  final 
phase  of  data  collection  for  IRR  occurred  simultaneously  with  the  testing  to  determine  known 
group  validity  (which  AMMP  test  tasks  could  distinguish  groups)  and  for  logistical  reasons, 


including  proximity  to  Fort  Bragg,  involved  only  2  raters,  both  physical  therapists.  This  group  of 
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26  subjects  was  tested  over  a  several  month  period  usually  involving  testing  one  or  two 
subjects  in  a  single  day. 

Data  Analysis  for  all  phases 

The  Krippendorf  Alpha  [15]  was  used  to  evaluate  inter-rater  reliability.  This  general  measure 
can  be  used  regardless  of  the  number  of  observers,  sample  size,  missing  data  and  type  of 
measurement  (nominal,  ordinal,  interval,  or  ratio).  For  both  interval  and  ratio  data  the  analysis 
is  equivalent  to  the  intraclass  correlation  coefficient  (ICC)  for  two  observers  and  is  extended  for 
many  observers.  For  nominal  data,  analysis  for  two  observers  is  equivalent  to  Scott's  Pi.  Parallel 
analyses  using  both  the  Krippendorf  and  Kappa  (2  observers)  have  produced  identical  results. 
The  code  was  integrated  into  SPSS  V18.0.  Bootstrapping  using  an  n=2000  was  used  to  produce 
95%  confidence  intervals.  In  some  cases  where  the  scorers  were  not  constant,  the  SPSS  V18.0 
ICC  analysis  using  the  two-way  random  model  was  used  to  confirm  the  Krippendorff  result.  For 
items  that  required  a  yes/no  response,  number  of  triplet  (or  couplet  in  Phase  III)  scoring 
disagreements  was  determined.  Given  that  a  full  range  of  combinations  of  responses  did  not 
occur,  a  Kappa-like  analysis  was  not  possible.  Target  ICC  was  set  at  >0.90  for  dual-tasks  and 
>0.85  for  multitasks  [16]  in  order  to  meet  clinical  expectations  for  reliability  of  assessments. 

In  addition  to  the  use  of  IRR  findings,  clinical  feasibility  (time  to  administer,  test  space 
requirements,  cost  and  durability  of  equipment)  and  both  verbal  and  written  feedback  from 
subjects  were  discussed  among  the  test  developers  and  used  to  drive  task  refinement.  The 
first  20  HC  subjects  tested  at  USARIEM  were  administered  a  team  developed  "experience 


14 


survey"  which  asked  for  opinions  on  the  clarity  of  instructions  and  difficulty  of  AMMP  tasks  in 
addition  to  requesting  general  comments  on  the  AMMP  test  battery.  This  feedback  from  test 
subjects  drove  edits  to  clarify  test  administration  instructions  and  provided  impressions  of  face 
validity  of  the  individual  tasks. 


RESULTS: 

The  revised  AMMP  tasks  and  example  modifications  are  described  in  Table  1.  Each  of  the  6 
tasks  uses  a  unique  scoring  system  (Table  1,  left  column)  related  to  observable  domains  and 
task  demands  [10].  Three  of  the  tasks  also  used  instrumented  measures  (inertial  sensors)  that 
were  not  evaluated  for  inter-rater  reliability  and  was  not  modified  during  this  task  refinement 
process. 

Not  all  subjects  completed  all  tasks.  In  order  to  avoid  exceeding  the  maximum  IRB  approved 
test  time,  subjects  who  were  slower  to  complete  tasks  were  not  asked  to  begin  the  last 
scheduled  task.  Some  subjects  did  not  complete  testing  due  to  overlapping  scheduling 
conflicts.  Four  (Run-Roll-Aim)  to  6  (Illinois  Agility-Packing  List)  subjects  with  mTBI  self-selected 
to  stop  or  were  stopped  by  the  primary  rater  before  completing  all  trials  due  to  an  exacerbation 
of  dizziness  or  headache  symptoms.  One  healthy  control  subject  was  stopped  from  completing 
all  trials  of  the  Illinois  Agility  Test-Packing  List  task  because  he  reported  a  mild  aggravation  to  a 
prior  ankle  injury. 


Phase  Mil: 
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As  previously  described,  findings  from  Phases  I  and  II  informed  iterative  improvements  to  the 
AMMP  tasks  [5],  (Figure).  After  these  initial  revisions,  the  remaining  6  test  tasks  of  the  AMMP 
battery  continued  through  this  refinement  process.  Results  of  the  iterative  process  and  phase- 
specific  IRR  will  be  summarized  first  for  the  dual-tasks  and  then  for  the  multitasks. 

Dual-task  Revisions 

Illinois  Agility-Packing  List:  Rater  reliability  for  run  time  and  words  recalled  correctly  for  the 
Illinois  Agility-Packing  List  (Table  2)  task  were  >  0.89  for  all  trials  in  healthy  control  subjects 
tested  in  Phase  I.  ICC  for  packing  list  errors  was  poor  (0.07  to  0.10)  in  this  HC  group.  Revisions 
of  scoring  rules  to  require  that  the  recalled  word  matched  exactly  the  given  word  (e.g. 
"bandage"  repeated  as  "band  aid"  defined  as  an  error)  and  revision  of  the  initial  packing  list  to 
replace  multiword  items  (e.g.  "100  mph  tape")  improved  the  ICCs  for  word  errors  when  initially 
tested  on  subjects  with  mTBI  (0.54  to  0.85  depending  on  trial)  though  still  below  an  acceptable 
range.  Further  revisions  of  the  packing  list  ensued,  however,  the  inability  of  multiple  raters  to 
accurately  hear  responses  appeared  to  contribute  to  lower  reliability. .  As  well,  subjects  were 
instructed  to  speak  with  sufficient  volume  and  raters  were  instructed  to  stand  in  close 
proximity  to  the  subject  at  the  end  of  the  agility  run.  ICC's  for  all  revised  metrics  were  above 
0.96  when  tested  on  the  final  23  subjects.  An  additional  metric  of  'course  error'  was  added  in 
order  to  capture  the  number  of  times  a  subject  made  an  error  in  navigating  the  agility  course 
correctly.  ICCs  for  this  metric  for  the  final  23  subjects  were  from  0.77  to  1.0  depending  on  trial, 


noting  that  many  subjects  made  no  errors  running  the  agility  course. 
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Load  the  Magazine-Radio  Chatter:  For  this  task,  all  metrics  remained  unchanged  throughout 
all  3  phases  of  reliability  testing.  The  ICCs  for  the  Load  Magazine-Radio  Chatter  Dual-task  (Table 
3)  were  greater  than  0.93  when  tested  on  healthy  control  subjects  in  Phase  I.  When  tested  on 
subjects  with  mTBI,  the  ICCs  for  distractor  words  dropped  (0.69  in  single  task,  0.50  in  dual  task 
condition).  Feedback  from  raters  indicated  that  in  the  testing  space  at  Fort  Bragg,  ambient 
sounds  interfered  and  not  all  subjects  spoke  loud  enough  to  hear  over  the  recorded  radio 
chatter  and  ambient  noise.  In  addition,  responses  were  sometimes  delayed  in  time  after  the 
target  word  was  spoken.  Revised  instructions  to  subjects  following  the  first  practice  trial  (if  a 
voice  volume  issue  was  identified)  were  to  "speak  loudly  so  we  can  all  hear  you"  and  to  require 
the  raters  to  be  seated  directly  in  front  of  the  subject.  Clarification  of  instructions  for  when  a 
target  response  was  provided  in  sufficient  time  to  be  counted  as  correct,  reduced  rater 
uncertainty  for  marking  a  response  correct  or  incorrect.  ICCs  for  the  final  reliability  testing 
were  all  greater  than  0.95  for  this  task. 

Instrumented  Stand  and  Walk-Grid  coordinates:  ISAW-grid  task  metrics  (Table  4)  remained 
unchanged  throughout  all  3  phases  of  reliability  testing.  IRR  was  generally  excellent  for  time 
and  grid  coordinate  measures  (ICCs  >  0.92)  in  HC  subjects  in  Phase  I.  Initial  testing  in  subjects 
with  mTBI  showed  a  drop  in  the  ICCs  (4  were  below  0.90)  which  when  later  discussed  among 
raters  appeared  to  be  the  result  of  some  raters  not  being  able  to  hear  the  subject  vocalize  the 
grid  coordinates  at  the  end  of  the  walk  and  a  greater  range  of  error  patterns  that  had  not  been 


observed  in  HC  subjects.  Changes  were  made  to  administration  instructions  to  require  raters  to 
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move  closer  in  proximity  to  the  subject  at  the  point  they  finished  their  walk  and  to  request  that 
subjects  speak  louder  to  facilitate  hearing  their  responses  over  any  ambient  sounds. 

Additional  clarification  of  scoring  rules  improved  the  ability  to  rate  these  responses  reliably. 
Final  testing  on  26  subjects  resulted  in  ICCs  >  0.92  for  all  metrics  in  the  ISAW-grid  task. 

Multitask  Revisions 

Patrol-Exertion  Task:  The  Patrol-Exertion  (Table  5)  task  underwent  multiple  revisions  in  its 
initial  format  as  a  SALUTE  report.  The  SALUTE  report  is  a  type  of  standard  Army  reconnaissance 
report  which  requires  specific  information  related  to  the  size,  activity,  location,  unit  or  uniform, 
time  and  equipment  (SALUTE)  of  the  observed  enemy.  ICCs  for  various  components  of  the  total 
SALUTE  report  and  the  total  score  ranged  from  0.29  to  0.89  in  testing  20  HC  subjects  (Table  5) 
during  Phase  la  and  lb.  The  addition  of  the  reporting  of  IED  markers  seen  during  scanning 
reports  for  Version  2  of  the  SALUTE  tested  on  12  HC  subjects  resulted  in  ICCs  that  ranged  from 
0.14  to  0.90.  Testing  with  7  subjects  with  mTBI  at  Fort  Bragg  was  insufficient  to  calculate  ICC, 
however,  disagreements  among  raters  were  evident  (Table  5).  Discussions  with  subjects,  with 
military  subject  matter  experts,  and  AMMP  developers  led  to  the  determination  that  while 
reconnaissance  reporting  is  used  within  the  military,  actual  reporting  of  intelligence  using  this 
SALUTE  format  tended  to  be  highly  variable  in  degree  of  detail,  ordering,  and  overall  content 
reported.  Those  subjects  with  combat  experience  were  more  likely  to  verbalize  their  simulated 
report  to  "higher  command"  outside  of  the  standard  SALUTE  format  prioritizing  brevity  and  key 
findings  over  the  longer,  more  detailed  doctrinal  SALUTE  format.  Reporting  format  varied 
greatly  between  those  subjects  with  mTBI  who  had  been  deployed  and  those  that  had  not. 
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Rank,  and  previous  experience  serving  in  key  leadership  positions  in  a  patrol  also  appeared  to 
affect  how  a  Soldier  reported  pertinent  information.  Following  initial  IRR  testing  at  Fort  Bragg, 
the  reporting  format  was  changed  to  a  general  post-patrol  question-answer  format  with  clear 
criteria  for  correct  and  incorrect  responses.  A  reaction  time  component  was  added  and 
questions  on  visual  clarity  and  perceived  exertion  (RPE)  during  stepping  were  also  added  to  the 
metrics  for  this  task.  The  ICCs  for  all  metrics  for  the  Patrol-Exertion  task  were  above  0.96  for 
the  final  26  subjects  tested  at  Fort  Bragg. 

Charge  of  Quarters  Duty  Task:  ICC  findings  for  CQ  Duty  (Table  6)  for  task  performance  (0.94), 
number  of  transits  between  work  stations  (0.98)  and  total  task  completion  time  (0.98)  were 
excellent  when  tested  on  12  HC  volunteers  at  USARIEM.  ICC  findings  for  the  number  of  rule 
breaks  was  0.66  (Table  5)  in  this  HC  population.  ICCs  for  all  metrics  except  rule  breaks  (0.35) 
were  clinically  acceptable  (0.90)  when  tested  on  12  subjects  with  mTBI  at  Fort  Bragg. 
Clarifications  to  operational  definitions  of  rule  breaks  and  inclusion  of  example  rule  breaks  on 
the  score  sheet  were  some  of  the  revisions  made  based  on  these  findings  and  on  rater 
feedback.  The  ICCs  for  all  metrics  for  the  CQ  Duty  task  were  above  .087  for  the  final  25  subjects 
tested  at  Fort  Bragg. 

Run-Roll-Aim  Task:  The  Run-Roll-Aim  task  (Table  7)  demonstrated  ICCs  of  0.50  to  0.99  (Table  7) 
depending  on  the  metric,  for  HC  subjects  tested  at  USARIEM.  The  ability  to  hear  a  subject's 
verbal  identification  of  visual  targets  was  problematic  in  the  early  testing  stage,  necessitating 


test  instructions  to  "speak  loudly  so  we  can  hear  you".  This  task  included  a  potential  error  of 
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"hesitation"  related  to  the  directional  Stroop  cue.  This  element  was  problematic  for  rating, 
given  with  an  incongruent  Stroop  cue  where  the  letter  for  right  (R)  or  left  (L)  roll  did  not  match 
the  directional  arrow,  some  delay  is  typical,  so  determining  whether  a  motor  delay  was  of 
inordinate  length  was  difficult.  The  individual  who  is  operating  the  remote  that  advances  the 
slides  that  presents  the  computer  cues  has  an  innate  sense  of  the  delay  post-Stroop 
presentation  that  observing  raters  likely  do  not.  Revisions  to  operational  definitions  of  errors 
were  the  primary  changes  made  to  this  task.  Improvement  in  ICC's  were  seen  when  testing  the 
final  26  subjects  at  Fort  Bragg  for  time  and  odd/even  numbers  (ICC  >  0.93)  reported.  The  ICC's 
for  total  errors  was  0.64  and  for  total  cues  was  0.87  in  this  group. 


DISCUSSION 

Rehabilitation  assessments  that  involve  multi-system  performance  often  require  a  multistep 
process  of  development  and  refinement  in  order  to  establish  IRR  [9-11,17,18].  Acceptable 
levels  of  IRR  were  achieved  for  the  AMMP  dual-task  and  multitask  metrics  with  the  application 
of  a  deliberate  refinement  process  that  recognized  the  importance  of  measure  reliability  and 
feasibility.  In  this  early  stage  of  the  AMMP  validation  process,  we  have  chosen  to  deal  with  the 
complexity  of  the  multifaceted  metrics  that  are  used  in  dual-task  and  in  multitask  measures,  by 
evaluating  rater  reliability  for  each  separate  task  metric.  As  the  AMMP  validation  process 
proceeds,  we  aim  to  normalize  performance  across  AMMP  tasks,  combining  individual 
component  metrics  to  generate  a  composite  score  for  each  task  and  potentially  for  the 
complete  AMMP  battery.  Composite  scores  should  ease  interpretation  and  facilitate  decision 


making  as  demonstrated  with  other  batteries  described  in  the  rehabilitation  literature  [10,19]. 
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The  AMMP  is  intended  for  use  in  combination  with  other  metrics  to  inform  return-to-duty 
decision-making  in  SM  with  mTBI.  To  make  RTD  recommendations,  the  importance  of  rater 
reliability  in  a  metric  cannot  be  overstated  [16,20].  Kottner  et.  al.,  suggests  that  when 
important  decisions  on  individuals  are  being  made  on  the  basis  of  an  assessment  score,  rater 
reliability  values  should  be  0.90  or  0.95  [20].  Not  all  metrics  for  the  AMMP  tasks  met  this 
stringent  standard  however,  following  the  iterative  process  in  the  AMMP  battery  development, 
the  majority  of  the  ICC's  were  above  0.90  (Tables  2-7),  supporting  the  continuation  of  the 
validation  process  for  the  component  tasks  in  this  assessment  battery.  Those  metrics  that  did 
not  meet  this  standard  were  typically  characterized  by  restricted  value  ranges  which  can 
significantly  reduce  ICC  values. 

The  process  used  for  refinement  of  AMMP  test  tasks  began  with  healthy  control  (HC)  subjects. 
Given  that  the  initial  9  tasks  took  approximately  3  hours  to  complete,  testing  on  HC  subjects 
allowed  our  team  to  recognize  tasks  that  lacked  practicality,  feasibility  and  face  validity  for 
healthy  active  duty  SM.  Testing  in  a  HC  group  provided  investigators  with  the  early 
opportunity  to  evaluate  the  level  of  difficulty  for  individual  test  tasks  among  SM  considered 
"duty-ready"  and  deployable  by  military  standards.  Reliability  metrics  for  four  of  the  6  retained 
tasks  (Tables  2-7)  were  at  or  above  clinically  acceptable  levels  in  this  HC  group.  These  findings 
were  consistent  with  the  rehabilitation  literature  wherein  rater  reliability  for  functional  tasks 
are  often  better  (higher)  for  patients  who  cluster  at  the  high  or  the  low  end  of  the  spectrum 


[21(page  6)]. 
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Phase  II  testing  on  subjects  with  mTBI  underscored  the  importance  of  optimizing  administration 
and  scoring  procedures  on  HC  subjects.  By  the  time  the  AMMP  battery  was  brought  to  Fort 
Bragg  for  testing  on  subjects  with  mTBI-related  residual  symptoms,  the  mean  battery  testing 
time  had  dropped  to  1  hour  and  45  minutes,  which  was  reasonably  well  tolerated  by  this  cohort 
of  participants.  With  the  introduction  of  subjects  with  mTBI  into  the  study  participant  pool, 
investigators  noted  a  drop  in  ICCs  for  several  of  the  AMMP  task  metrics  (Tables  2-7)  that  had 
previously  been  at  or  close  to  clinically  acceptable  levels.  Subjects  endorsing  mTBI-related 
symptoms  committed  more  frequent  and  wide  ranging  errors  on  AMMP  tasks  than  were  not 
recorded  in  testing  HC  Soldiers  at  USARIEM.  These  novel  errors  required  research  team 
members  to  operationally  define  metrics  explicitly,  clarifying  acceptable  and  unacceptable 
responses.  The  initial  groups  of  subjects  with  mTBI  also  represented  a  broad  range  of  Soldier 
ranks,  occupational  specialties  and  deployment  experiences  which  likely  contributed  to  the 
variety  of  participant  responses.  In  addition  to  the  formal  test  responses,  the  feedback  on  the 
task  expectations  and  realism  of  the  test  metrics  was  made  clear  to  our  research  team  during 
this  process  and  contributed  to  practical  aspects  of  task  refinement. 

Modifications  to  several  AMMP  tasks  underscored  lessons  learned  and  the  modifications  that 
were  required  to  achieve  clinically  acceptable  reliability.  Development  of  the  lAT-Packing  List 
task  highlighted  the  importance  of  definitive  rules  for  giving  credit  for  a  correct  answer.  During 
early  reliability  testing,  some  raters  gave  credit  for  returned  words  that  were  "close  but  not 
exact"  such  as  accepting  the  word  "band  aid"  when  the  word  given  was  "bandage".  When  this 


"benefit  of  the  doubt"  scoring  was  used  by  some  but  not  all  raters,  the  IRR  suffered.  Packing 
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lists  were  modified  to  reduce  ambiguous  or  easily  misunderstood  words.  Scoring  rules  were 
clarified  to  require  verbatim  word  responses  from  the  subject,  resulting  in  improved  ICCs  to 
clinically  acceptable  levels.  Difficulty  with  hearing  and  perceiving  items  from  the  packing  list 
occurred  more  often  when  test  subjects'  first  language  was  not  English.  Having  subjects  repeat 
the  words  as  they  are  provided  could  reduce  the  likelihood  of  misunderstanding  based  on 
accent  or  language  differences  of  the  subject  or  the  tester,  but  this  was  not  done  in  this  study. 

Other  real-world  lessons  learned  reinforced  the  practical  requirements  for  reliable  scoring  of 
tests  that  entail  recording  subject  performance,  as  well  as  their  verbal  responses.  Performance- 
based  assessments  with  verbal  responses  require  appropriate  rater  vantage  point  to  clearly 
hear  the  subject's  responses.  If  more  than  1  rater  was  scoring,  the  physical  set-up  for  test 
administration  must  be  conducive  to  allow  all  raters  the  ability  to  hear  verbal  responses.  The 
issue  of  being  able  to  hear  a  verbal  response  may  be  less  of  a  concern  for  clinical  use  of  a 
measure,  given  the  test  administrator  is  typically  in  the  best  position  to  hear  a  patient's 
response  and  usually  is  the  sole  rater  of  performance. 

Development  of  the  Load  Magazine-Radio  Chatter  dual-task  and  the  Patrol-Exertion  multitask 
highlighted  the  importance  of  quantifiable  metrics  that  were  objective  and  non-ambiguous 
requiring  no  subjective  interpretation  on  the  part  of  the  examinee  or  the  rater.  For  example,  in 
the  early  development  phase  of  the  radio  chatter,  instructions  were  to  respond  to  chatter  that 
was  "relevant"  to  a  specific  character  in  the  chatter.  Relevance  was  not  sufficiently  defined  and 
was  interpreted  very  broadly  by  subjects  during  pilot  testing.  In  the  final  version  of  this  task. 
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participants  were  instructed  to  respond  affirmatively  to  the  recognition  of  pre-established  words  like 
"break"  from  specific  character  voices  in  the  pre-recorded  radio  chatter  which  eliminated  ambiguity 
regarding  correct  and  incorrect  responses. 

Similarly,  early  versions  of  the  PATROL-Exertion  task  required  reporting  using  a  standard 
reconnaissance  SALUTE  report  of  the  sort  used  during  military  operations.  During  several 
rounds  of  reliability  testing  and  repeated  modifications  to  subject  instructions  and  scoring 
examples,  answers  for  this  type  of  reconnaissance  report  varied  and  were  clearly  based  on 
judgment  of  the  subject  using  their  deployment  experience,  military  occupational  specialty  and 
rank.  Raters  also  frequently  used  "benefit  of  the  doubt"  scoring  depending  on  the  rater's  own 
experience.  Final  modifications  required  post-patrol  reporting  that  had  unmistakably  defined 
answers  to  questions  such  as  "What  weapon(s)  did  you  see"?  This  type  of  question-response 
format  required  focused  attention  to  the  Patrol  task  video  without  necessitating  judgments  by 
the  subject  or  the  rater.  This  change  facilitated  the  clear  cut  scoring  of  subject  responses  by  the 
raters.  Sapsford  and  Jupp  discuss  issues  with  observational  research  including  problems  with 
"inconsistency  in  the  way  rules  are  applied  by  different  observers  and  sometimes  by  the  same 
observer  on  different  occasions"  [22(page  70)].  This  inconsistency  is  often  seen  when 
inferences  for  scoring  or  coding  behavior  are  required  and  when  there  are  ambiguities  in  the 
scoring  system  [22].  These  examples  clarify  the  importance  of  using  well  validated  tests  with 
clear  cut  metrics.  As  well,  they  underscore  the  fallibility  of  make-up-your-own  clinical  tests 
involving  dual-task  and  multitask  activities  and  the  importance  of  a  standardized  test  that  has 


established,  acceptable  rater  reliability. 
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One  side  benefit  of  this  repeated  process  of  reliability  testing  and  task  material  revision  is  the 
improved  and  succinct  administration  instructions  and  the  script  for  administering  the  tasks  to 
subjects.  Firsthand  participation  in  at  least  several  testing  sessions  by  all  members  of  the 
AMMP  development  team  helped  to  maximize  the  feedback  for  revisions[10,ll].  Our  initial 
target  has  been  to  provide  an  assessment  tool  for  rehabilitation  professionals  including  physical 
and  occupational  therapists.  One  future  goal  is  to  work  towards  an  assessment  battery  that 
may  be  appropriately  used  by  other  military  medical  providers  such  as  nurse  practitioners, 
physician's  assistants,  and  medics.  This  goal  will  require  further  examination  of  training 
requirements  and  additional  reliability  testing  to  compare  administration  of  the  AMMP  by 
clinicians  who  commonly  interact  with  SM  to  a  "gold  standard"  of  trained  PT  and  OT  raters. 

The  amount  of  training  required  to  achieve  clinically  acceptable  reliability  has  yet  to  be  clarified 
[21(page  10)]. 

One  of  this  study's  strengths  was  the  iterative  process  used  that  resulted  in  clinically  acceptable 
inter-rater  reliability  for  this  version  of  the  AMMP.  Additionally,  the  iterative  process  facilitated 
the  development  of  succinct  instructions  to  the  subjects,  as  well  as  efficient  scoring  forms.  Our 
final  inter-rater  reliability  findings  were  sufficient  to  support  continued  validation  of  this 
assessment  tool. 

This  study  has  a  number  of  limitations.  Despite  the  defined  inclusion  criteria,  no  subjects  with 


mTBI  residual  symptoms  were  less  than  2  months  post  most  recent  concussion.  Scoring  of  and 
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responses  from  subjects  with  more  acute  symptoms  and  from  populations  at  additional 
installations  or  deployment  environments  may  result  in  a  requirement  for  further  refinement  of 
operational  definitions  of  task  metrics  including  expansion  of  acceptable  and  unacceptable 
answers.  All  raters  for  this  study  were  physical  and  occupational  therapists  with  a  minimum  of 
6  years  of  experience  and  a  knowledge  of  the  background  and  development  process  of  the 
AMMP  battery.  This  may  have  contributed  to  a  bias  in  scoring  some  or  all  of  the  AMMP  tasks. 
Further  reliability  testing  with  novice  raters  who  did  not  participate  in  the  development  of  this 
assessment  will  contribute  to  clarity  on  the  amount  of  training  required  to  achieve  adequate 
IRR  for  a  clinical  metric.  The  practical  feedback  received  from  the  development  team  and  from 
the  subjects  resulted  in  clinically  feasible  AMMP  tasks  with  some  degree  of  face  validity.  These 
tasks,  however,  may  not  be  feasible  in  all  test  environments  as  some  of  the  tasks  require  a 
relatively  large  space  or  a  very  quiet  test  environment  without  ambient  distractions.  This  will 
restrict  the  use  of  specific  AMMP  tasks  to  environments  with  adequate  facilities.  However,  the 
strong  face  validity  and  functional  relevance  likely  outweigh  the  environmental  constraints  of 
the  testing  environments. 

CONCLUSIONS:  The  AMMP  was  developed  to  fulfill  a  defined  need  for  a  performance  based 
assessment  following  military  concussion.  Military  stakeholder  requirements  for  face  validity, 
and  functional  relevance  contribute  to  the  complexity  of  development  of  a  reliable  AMMP 
battery  as  a  performance-based  assessment  evaluating  multiple  domains  of  function.  The 
consistency  of  scores  across  raters  is  fundamental  to  the  ability  to  use  the  findings  of  the 
AMMP  to  make  substantive  recommendations  regarding  readiness  to  RTD  following 
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concussion/mTBI.  This  research  demonstrated  that  following  an  iterative  development 
process,  individual  AMMP  tasks  appear  feasible,  and  have  metrics  that  can  be  reliably  scored  by 
experienced  rehabilitation  professionals.  Evaluation  of  preliminary  known  groups  and 
convergent  validity  using  correlation  to  standard  neurocognitive  tests  is  currently  underway 
with  members  of  the  AMMP  development  team.  Before  the  AMMP  is  used  clinically  to  inform 
RTD  decision-making,  further  evaluation  of  intra-rater,  novice  rater,  test-retest  reliability,  and 
additional  validation  studies  should  be  carried  out.  Future  work  to  further  clarify  discriminant 
and  convergent  validity  in  subjects  along  the  continuum  of  recovery  from  concussion  as  well  as 
evaluation  of  the  responsiveness  of  the  AMMP  should  be  undertaken. 
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FIGURE:  AMMP  Refinement  Phases 


Phases  la  USARIEM  (20  HC),  lb  USARIEM  (12  HC) 


Phases  lla  FORT  BRAGG  (12  mTBI),  Mb  FORT  BRAGG  (7  mTBI) 


Phases  III  FORT  BRAGG  (19  mTBI,  7  HC) 

ISAW-Grid 

Illinois  Agility-Packing  List 
Load  Magazine-Radio  Chatter 
Run-Roll-Aim 
Patrol-Exertion 
CQ  Duty 
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Table  1. 

The  Assessment  of  Military  Multitasking  Performance  Tasks,  metrics  and  example  modifications 


Dual-tasks 

Description 

Example  Modifications 

(Metrics) 

Illinois  Agility  Test 

The  IAT  requires  running  a  course 

i) 

Instruct  raters  to  stand  close  to 

(lAT)-Packing  List 

with  rapid  direction  changes  and 

and  in  front  of  subject  to  improve 

serpentine  navigation  of  4  cones.  In 

ability  to  hear  call  out  of  recalled 

-Run  time  (sec) 

the  dual  task  condition,  the  subject  is 

"packing  list"  words. 

-Words  correct 

given  a  5  to  7  word  packing  list  to 

2) 

Edit  packing  list  words  to  remove 

-Word  errors 

remember  (number  of  words  scaled 

easily  confused  (band  aid  vs 

-Path  errors 

to  number  recalled  in  single  task 

bandage)  or  3  word  items  (100 

-Dual  Task  Cost 

condition). 

mph  tape) 

(DTC) 

Load  Magazine- 

The  subject  loads  M-16  dummy 

1) 

Instruct  subject  to  "speak  loudly  so 

Radio  Chatter 

rounds  in  a  magazine  as  fast  as 

1  can  hear  you". 

possible.  The  dual-task  condition 

2) 

Instruct  raters  to  sit  directly  in 

-Rounds  loaded 

requires  monitoring  radio 

front  of  subjects  to  improve  ability 

DTC 

communication  about  an  upcoming 

to  hear  the  "check"  word  response 

-Words  Correct 

company  FTX ,  responding  "check" 

by  subject. 

-Word  Errors 

when  key  words  are  spoken  by 

-Words  DTC 

specific  characters  interacting  in  the 

radio  chatter. 

Instrumented 

The  ISAW-Grid  task  involves  using 

1) 

Instruct  subject  to  "speak  loudly  so 

Stand  and  Walk- 

wireless  wearable  inertial  sensors  and 

1  can  hear  you". 

Grid  Coordinates 

a  clinical  software  program  to 

2) 

Instruct  raters  to  stand  close  to 

(ISAW-Grid): 

measure  static  postural  sway  and 

and  in  front  of  subject  to  improve 

then  dynamic  stability  during  walking 

ability  to  hear  call  out  of  recalled 

-Walk  time  (sec) 

and  turning,  the  walk  is  timed.  A  grid 

grid-coordinates. 

-Grid  Coordinates 

memorization  task  provided  in  the 

correct 

context  of  a  patrol  mission  provides 

-DTC  time  and 

the  cognitive  challenge. 

correct  grid 

coordinates 

Multitasks 

Description 

Example  Modifications 

(Metrics) 

Patrol-Exertion 

The  subject  gathers  information  from 

i) 

SALUTE  video  revised  to  include 

(SALUTE  format 

video  surveillance  and  radio 

IED  marker  identification  to 

removed) 

communications  while  exercising  at 

increase  task  complexity. 

65%  to  85%  of  age  predicted  maximal 

2) 

Addition  of  auditory  reaction  time 

-IED  Markers 

heart  rate  by  doing  continuous  step- 

component. 

Correct 

ups  on  an  exercise  step  to  simulate  a 

3) 

Removal  of  "SALUTE"  reporting 

-Patrol 

dismounted  patrol.  IED  markers  and 

format  with  modification  to  a 

Observations 

pertinent  logistical  information  must 

"question-answer"  general 

correct 

be  recalled  and  reported  at  specific 

PATROL  reporting  format. 

-Visual  Clarity  (1- 

times  while  also  requiring  a  reaction 

4) 

Provide  specific  examples  on  score 

10  VAS) 

time  trigger  switch  press  to  an 

sheets  as  to  correct  answers  to 

PATROL  questions. 
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-RPE  (Borg  6-20 
scale) 

-Reaction  time 
(msec) 

intermittently  recurring  tone  sound 
(11  tones  in  12  minutes). 

5) 

Addition  of  symptom  report  for 
visual  clarity  and  level  of  effort 
(RPE). 

Charge  of  Quarters 

Requires  the  subject  to  organize  and 

1) 

Initial  revisions  described 

(CQ)  Duty 

implement  a  plan  in  order  to 

previously  (Smith  et  al.,  2014) 

complete  a  number  of  tasks  all  while 

2) 

Revisions  of  score  sheets  to 

-Task  Completion 

pulling  CQ  duty.  Tasks  such  as 

include  operational  definitions  for 

-Transits 

assembling  a  footstool,  inventorying 

all  task  performance  metrics  and 

-Total  Task  Time 

supplies,  radioing  barracks  room 

operational  definitions  of  errors. 

(min/sec) 

availability  to  the  1st  Sergeant  all 

3) 

Addition  of  several  task 

-Rule  Breaks 

occur  under  time  and  efficiency  rules. 

performance  items. 

Run-Roll-Aim 

The  RRA  task  requires  the  subject  to 

1) 

Instruct  subject  to  "call  out  the 

(RRA) 

respond  to  a  directional  Stroop  signal 

numbers  loudly  so  1  can  hear  you". 

and  complete  several  maneuvers  such 

2) 

Instruct  raters  to  stand  close  to 

-Task  completion 

as  avoiding  a  trip  wire,  a  3-5  second 

the  subject  while  s/he  is  calling  out 

time  (sec) 

rush,  combat  rolls,  side  shuttling  and 

numbers  to  improve  ability  to  hear 

-Correct  odd/even 

back  pedaling  while  carrying  a 

responses. 

numbers 

simulated  weapon.  The  subject  uses 

3) 

Improved  operational  definitions 

-Total  errors: 

a  short  focal  point  scope  on  the 

of  specific  errors  (i.e.. 

odd/even  number 

weapon  to  identify  numbers  on  a 

"hesitation"). 

and  sequence 

computer  screen  based  on 

4) 

Combination  of  errors  into  one 

instructions  given  before  the  task 

metric  so  that  all  errors  were 

starts. 

summed. 
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Table  2. 

Illinois  Agility-Packing  List  Interrater  Reliability 


Scoring  item 

USARIEM* 

n=20.  Healthy  Controls  (HC) 

Fort  Bragg/WAMC* 
n=12  SM  with  mTBI 

Fort  Bragg/WAMC# 
n=  23  (18  mTBI,  5  HC) 

(Metrics) 

Reliability,  ICC 

95%  Cl  (lower,  upper) 

Reliability,  ICC 

95%  Cl  (lower,  upper) 

Reliability,  ICC 

95%  Cl  (lower,  upper) 

Single  Task  Time 

0.98  (0.96-0.99) 

.82  (0.58-0.98) 

0.99  (0.98-0.99) 

Single  Task  Words  Correct 

0.89  (0.73-0.99) 

0.80  (0.69-0.90) 

1.0  (1-1) 

Single  Task  Word  Errors 

NA 

0.54  (0.12-0.83) 

1.0  (1-1) 

Dual  Task  No  Instruction:  Time 

0.96  (0.94-0.98) 

0.96  (0.93-0.98) 

0.99  (0.987-0.995) 

Dual  Task  Nl:  Words  Correct 

0.93  (0.88-0.97) 

0.93  (0.86-0.99) 

0.99  (0.97-1) 

Dual  Task  Nl:  Word  Errors 

0.07  (0-0.30) 

0.93  (0.87-0.97) 

0.99  (0.96-1) 

Dual  Task  Nl:  Course  Errors 

NA 

NA 

1.0  (1-1) 

Dual  Task  COG:  Time 

0.90  (0.85-0.95) 

0.98  (0.97-0.99) 

0.99  (0.989-.997) 

Dual  Task  COG:  Words  Correct 

1.0  (1.0-1.0) 

0.97  (0.92-1) 

1.0  (1-1) 

Dual  Task  COG:  Word  Errors 

0.10  (0.0-0.36) 

0.74  (0.37-0.99) 

0.996  (0.987-1) 

Dual  Task  COG:  Course  Errors 

NA 

NA 

0.77  (0-1) 

Dual  Task  MOB:  Time 

0.96  (0.91-0.98) 

0.88  (0.80-0.95) 

0.98  (0.97-0.99) 

Dual  Task  MOB:  Words  Correct 

1.0 (1-1) 

1.0  (1-1) 

1.0  (1-1) 

Dual  Task  MOB:  Word  Errors 

0.09  (0-0.35) 

0.85  (0.64-1) 

0.86  (0.58-1) 

Dual  Task  MOB:  Course  Errors 

NA 

NA 

1.0  (1-1) 

COG:  Cognitive  priority;  "concentrate  on  remembering  the  words",  Nl:  no  instruction  given,  MOB:  Mobility  priority; 
"concentrate  on  going  as  fast  as  you  can"  NA:  not  applicable  or  not  evaluated 
*  =  3  raters;  #  =  2  raters 
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Table  3. 

Load  Magazine-Radio  Chatter  Interrater  Reliability 


Scoring  item 
(Metrics) 

USARIEM* 

n=20  Healthy  Controls  (HC) 

Fort  Bragg/WAMC* 
n=12  SM  with  mTBI 

Fort  Bragg/WAMC# 
n=  24  (18  mTBI,  6  HC) 

ICC  (95%  Cl) 

ICC  (95%  Cl) 

ICC  (95%  Cl) 

Rounds  loaded  single  &  dual® 
Correct  Key  Word  Single 
Distractor  Key  Word  Single 
Correct  Key  Word  Dual 
Distractor  Key  Wo 
rd  Dual 

Not  evaluated 

0.99  (0.98-1) 

0.93  (0.83-1) 

0.98  (0.96-1) 

0.97  (0.95-0.99) 

Not  evaluated 

0.94  (0.88-0.99) 
0.69  (0.38-0.92) 
0.99  (0.97-1.0) 

0.50  (0.11-0.82) 

Not  evaluated 

0.997  (0.993-1) 
0.995  (0.986-1) 
0.978  (0.949-0.999) 

0.947  (0.869-1) 

*  =  3  raters;  #  =  2  raters;  @  =  for  practical  reasons,  all  dummy  rounds  were  counted  one  time,  not  by 
individual  rater 
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Table  4. 


Instrumented  Stand  and  Walk-Grid  Coordinates  (ISAW-grid) 


Scoring  item 

USARIEM* 

n=20,  Healthy  Controls  (HC) 

Fort  Bragg/WAMC* 
n=10  SM  with  mTBI 

Fort  Bragg/WAMC  # 
n=  26  (19  mTBI,  7  HC) 

(Metrics) 

ICC  (95%  Cl) 

ICC  95%  Cl 

ICC  (95%  Cl) 

Walk  Time  1  Single 

0.99  (0.98-1), 

0.77  (0.64-0.86) 

0.97  (0.96-0.99) 

Walk  Time  2  Single 

0.97  (0.96-0.98) 

0.95  (0.92-0.98) 

0.95  (0.90-0.98) 

Walk  Time  3  Single 

0.98  (0.97-0.99) 

0.91  (0.85-0.96) 

0.98  (0.97-0.99) 

Walk  Time  1  Dual 

0.92  (0.85,0.97) 

0.89  (0.78-0.98) 

0.92  (0.86-0.97) 

Walk  Time  2  Dual 

0.98  (0.97,0.99) 

0.94  (0.89-0.96) 

0.95  (0.90-0.98) 

Walk  Time  3  Dual 

0.97  (0.95-0.98) 

0.81  (0.72-0.88) 

0.98  (0.97-0.99) 

Grid  Coord  Single 

0.56  (0.14-0.90) 

0.88  (0.78-0.97) 

0.97  (0.92-1) 

Grid  Coord  1  Dual 

0.94  (0.90-0.97) 

0.94  (0.85-1) 

0.98  (0.93-1) 

Grid  Coord  2  Dual 

0.99  (0.97-1) 

0.99  (0.99-1) 

0.999  (0.998-1) 

Grid  Coord  3  Dual 

0.92  (0.84-0.99) 

1.0 (1-1) 

0.997  (0.990-1) 

*  =  3  raters;  #  =  2  raters 


37 


Table  5. 
Patrol-Exertion 


SALUTE-EXERTION  Version  1  (VI) 

SALUTE-EXERTION  Version  2  (V2) 

PATROL-EXERTION 

Scoring  item 

USARIEM* 
n=20  (VI) 

USARIEM* 
n=12 (V2) 

Fort  Bragg/WAMC* 
n=7  SM  with  mTBI  (V2) 

Scoring  item 

Fort  Bragg/WAMC" 
n=  26  (19  mTBI,  7  HC) 

(Metrics) 

Reliability,  ICC 

95%  Cl  (lower,  upper) 

(Metrics) 

Reliability  ICC 

95%  Cl  (lower,  upper) 

Size 

0.85  (0.76-0.93) 

0.72  (0.53-0.87) 

3  triplets  disagreed1 

X.  Sum  of  IED  markers 

0.95  (0.91-0.98) 

Activity 

0.29  (0.0-0.60) 

0.77  (0.58-0.94) 

5  triplets  disagreed1 

Y.  Sum  of  post-test 
patrol  questions 

0.97  (0.94-1) 

Location 

0.80  (0.64-0.93) 

0.78  (0.56-0.95) 

1  triplet  disagreed1 

Z.  Sum  of  X  and  Y 

0.97  (0.95-0.99) 

Unit 

NA@ 

0.14  (0-0.57) 

3  triplets  disagreed1 

Vision  clarity  initial 

0.99  (0.97-1) 

Time 

0.57  (0.22-0.86) 

0.73  (0.44-0.96) 

1  triplet  disagreed1 

Vision  clarity  end 

0.99  (0.98-1) 

Equipment 

0.89  (0.79-0.92) 

0.81  (0.62-0.95) 

3  triplets  disagreed1 

RPE  initial 

0.98  (0.95-1) 

Scan  IED  Markers  NA 

Total  Score  0.80  (0.66-0.91) 

0.90  (0.76-0.98) 
0.79  (0.66-0.90) 

0.97  (0.94-0.99) 

0.91  (0.84-0.96) 

RPE  end 

1.0  (1-1) 

@  =  In  the  initial  version,  the  report  was  described  as  a  SALTE  report  as  the  "Uniform  or  Unit"  component  of  the  report  was  not 
consistently  used,  based  on  advice  from  one  military  advisor  per  local  reporting  format. 

1  Due  to  insufficient  n  to  calculate  ICC,  the  number  of  rater  disagreements  is  reported 

RPE  =  Rate  of  Perceived  Exertion;  NA=not  applicable  or  not  evaluated 
*  =  3  raters;  #  =  2  raters;  Vl=Version  1,  V2=Version  2 
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Table  6. 

Charge  of  Quarters  (CQ)  Duty 


Scoring  item  n_ 

USARIEM  * 

12  Healthy  Control  (HC) 

Fort  Bragg/WAMC* 
n=12  SM  with  mTBI 

Fort  Bragg/WAMC# 
n=  25  (19  mTBI,  6  HC) 

(Metrics) 

ICC  (95%  Cl) 

ICC  (95%  Cl) 

ICC  (95%  Cl) 

Task  performance 

0.94  (0.86-0.99) 

0.90  (0.84-0.95) 

0.88  (0.76-0.97) 

#  of  Rule  breaks 

0.64  (0.32-0.90) 

0.46  (0.0-0.79) 

0.91  (0.75-1) 

#  of  Visits 

0.98  (0.96-0.99) 

0.92  (0.80-0.99) 

0.98  (0.97-0.99) 

Total  time 

0.98  (0.96-0.99) 

0.99  (0.99-1.0) 

0.998  (0.994-1) 

*  =  3  raters;  #  =  2  raters 
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Table  7. 


Run-Roll-Aim  (RRA) 


USARIEM* 

Fort  Bragg/WAMC* 

Fort  Bragg/WAMC  * 

n=20  Healthy  Controls  (HC) 

Scoring  item 

n=ll  SM  mTBI@ 

n=  26  (19  mTBI,  7  HC) 

Scoring  item 

ICC  (95%  Cl) 

(Metrics) 

Reliability,  ICC 

95%  Cl  (lower,  upper) 

Reliability,  ICC 

95%  Cl  (lower,  upper) 

Trial  1  incongruent 
numbers  correct 

0.996  (0.99-1) 

Trial  l-Time(secs) 

0.91  (0.80-0.99) 

0.999  (0.997-1) 

Trial  1  incongruent 
number  errors 

2  of  20  triplets 
disagreed 

Trial  1-numbers 

correct 

0.54  (0.08-0.89) 

0.96  (0.91-1) 

Trial  2  congruent 
numbers  correct 

0.86  (0.65-1) 

Trial  2-Time  (secs) 

0.80  (0.57-0.97) 

0.999  (0.993-0.999) 

Trial  2  congruent 
number  errors 

2  of  20  disagreed 

Trial  2-numbers 

correct 

0.55  (0.0-0.93) 

0.93  (0.70-1) 

Trial  3  congruent 
numbers  correct 

0.57  (0.15-0.89) 

Trial  3-Time(secs) 

0.86  (0.67-0.98) 

0.995  (0.991-1) 

Trial  3  congruent 
number  errors 

2  of  20  disagreed 

Trial  3-numbers 

correct 

0.72  (0.40-0.95) 

0.996  (0-1) 

Trial  4  incongruent 
numbers  correct 

0.50  (0.23-0.74) 

Trial  4-Time(secs) 

0.89  (0.75-0.98) 

0.999  (0.998-1) 

Trial  4  incongruent 
number  errors 

1  of  20  disagreed 

Trial  4-numbers 

correct 

0.99  (0.97-1.0) 

0.98  (0.96-1) 

Total  errors  (all 
trials) 

Total  cues  (all  trials) 


ICC's  for  individual  trials 
calculated,  Tl:  0.54,  T2:  0.13, 
T3:  0.18,  T4:  0.85 
ICC's  for  individual  trials 
calculated,  Tl:  0.71,  T2:  0.56, 
T3:  0.37,  T4:  NA& 


0.64  (0.13-0.92) 

0.87  (0.66-1) 


Time  to  complete  the  RRA  was  not  scored  by  all  raters  during  initial  testing. 

Errors  for  HC  were  recorded  as  #triplets  disagreed  as  the  range  of  errors  was  low,  ie.,  0-3  total; 

Prior  to  testing  mTBI,  revisions  were  made  to  score  sheets  and  instructions;  -Congruent  directional  Stroop  cue=  roll  direction 
arrow  and  R/L  letter  match,  incongruent  directional  Stroop  cue  =  roll  direction  arrow  and  R/L  letter  do  not  match: 

Correct  /  Errors  =  odd  or  even  numbers  viewed  through  scope  and  called  out:  &  No  cues  required— all  zeros. 

*=3  raters;  #=2  raters;  @  =  not  all  subjects  were  able  to  tolerate  completion  of  all  trials; 
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